apache/beam · error · SchemaDoesntMatchException

Problem converting field %s expected type: %s

Error message

Problem converting field %s expected type: %s

What it means

During field conversion, any exception while converting a TableRow value to the schema-declared type is caught and rethrown as SchemaDoesntMatchException("Problem converting field <name> expected type: <type>", cause). It signals the row's value cannot be coerced into the declared BigQuery field type (e.g. string into INT64, wrong nested record shape).

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java:1006

          Preconditions.checkArgumentNotNull(builder).setField(fieldDescriptor, value);
        }
        // For STRUCT fields, we add a placeholder to unknownFields using the getNestedUnknown
        // supplier (in case we encounter unknown nested fields). If the placeholder comes out
        // to be empty, we should clean it up
        if ((fieldSchemaInformation.getType().equals(TableFieldSchema.Type.STRUCT)
                && unknownFields != null)
            && ((unknownFields.get(key) instanceof Map
                    && ((Map<?, ?>) unknownFields.get(key)).isEmpty()) // single struct, empty
                || (unknownFields.get(key)
                        instanceof List // repeated struct, empty list or list with empty structs
                    && (((List<?>) unknownFields.get(key)).isEmpty()
                        || ((List<?>) unknownFields.get(key))
                            .stream()
                                .allMatch(row -> row == null || ((Map<?, ?>) row).isEmpty()))))) {
          unknownFields.remove(key);
        }
      } catch (Exception e) {
        throw new SchemaDoesntMatchException(
            "Problem converting field "
                + fieldSchemaInformation.getFullName()
                + " expected type: "
                + fieldSchemaInformation.getType(),
            e);
      }
    }
    if (changeType != null) {
      Preconditions.checkArgumentNotNull(builder)
          .setField(
              Preconditions.checkStateNotNull(
                  Preconditions.checkArgumentNotNull(descriptor)
                      .findFieldByName(StorageApiCDC.CHANGE_TYPE_COLUMN)),
              changeType);
      Preconditions.checkArgumentNotNull(builder)
          .setField(
              Preconditions.checkStateNotNull(
                  Preconditions.checkArgumentNotNull(descriptor)

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the wrapped cause 'e' — it names the exact conversion failure; fix the value format at the source.
2. Validate/coerce values against the schema (e.g. parse numerics, normalize timestamps to ISO-8601) before writing.
3. Correct the table schema if the new value type is intentional (e.g. widen INT64 to NUMERIC/STRING).
4. Catch SchemaDoesntMatchException in the write's error handling and route bad records to a dead-letter sink.

Example fix

// before
row.set("count", "12abc"); // schema INT64 -> SchemaDoesntMatchException
// after
row.set("count", Long.parseLong(countStr.trim())); // or validate upstream

// Pre-write coercion against schema
Object v = row.get(name);
if ("INTEGER".equals(fieldType) && v instanceof String) {
  row.set(name, Long.parseLong((String) v));
}

try {
  writeResult = rows.apply("WriteBQ", BigQueryIO.writeTableRows()...);
} catch (Exception e) {
  Throwable root = e;
  while (root.getCause() != null) root = root.getCause(); // find conversion cause
  LOG.error("Field type mismatch: {}", root.getMessage());
  throw e;
}

When it happens

Trigger: Writing a TableRow via the Storage API where a field's runtime value type doesn't match the schema type: non-numeric string for an INTEGER field, malformed timestamp, list where a scalar is required, nested record with wrong sub-shape.

Common situations: Upstream data quality issues (nulls/strings in numeric columns), schema changed to a stricter type, timestamps not in expected format, JSON-sourced rows with heterogeneous types.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• The input schema must have exactly one field of type byte.
• Cannot merge two types: +fieldType1.getTypeName()+ and +fiel
• value type is '%s' for field type '%s'
• Unreachable case for Beam typename %s
• Bounded Source is not BigQueryStorageStreamSource, unable to

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0c02f0b22373390b. Report an issue: GitHub.