apache/beam · error · SchemaDoesntMatchException

Could convert schema for

Error message

Could convert schema for %s

What it means

Same wrap-as-SchemaDoesntMatchException pattern as error 1840 but on the map-based conversion path (messageFromMap / TableRow to proto for map-like rows): the final DynamicMessage could not be built for the given schema, or collected per-field exceptions were rethrown here. The table name is included in the message.

Solutions

1. Inspect the exception cause for the missing/invalid field name.
2. Ensure all required fields are set before writing the row.
3. Regenerate the proto descriptor from the current table schema (refresh with StorageApiDynamicDestinations / table schema cache).
4. Filter or repair bad rows upstream with a validation transform before the sink.

Example fix

// before
builder.build() fails because 'id' is REQUIRED but absent
// after
if (row.get("id") == null) { throw new IllegalArgumentException("id required"); } row.set("id", id);

if (!requiredFields.stream().allMatch(f -> row.get(f) != null)) { throw new IllegalArgumentException("row missing required fields"); }

boolean rowFulfillsRequired(Map<String,Object> row, Descriptor protoDescriptor) { return protoDescriptor.getFields().stream().filter(f -> f.isRequired()).allMatch(f -> row.get(f.getName()) != null); }

try { builder.build(); } catch (SchemaDoesntMatchException e) { retryWithRepairedSchema(row, e.getCause()); }

When it happens

Trigger: Converting a TableRow (map of field name to value) into the Storage API proto when required proto fields are unset or a nested conversion recorded exceptions, causing builder.build() to throw (e.g. missing required field).

Common situations: Rows produced by a different pipeline stage with looser validation; CDC/schema-evolution jobs writing rows against a stale schema; Beam SQL rows cast to TableRows losing nullability information.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Converting BigQuery type
• Couldn't convert schema for
• Error while converting table schema
• Field is not nullable.
• Nested ROW missing row schema

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7ac3454c15b5af97. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java:1201

      }

      // If there are unknown fields, copy them into the output.
      if (unknownFields != null) {
        for (int i = cellsToProcess; i < cells.size(); ++i) {
          // TableCell accepts a null value, but the API client is not annotated.
          @SuppressWarnings("nullness")
          TableCell unknownCell = new TableCell().setV(cells.get(i).get("v"));
          unknownFields.getF().set(i, unknownCell);
        }
      }

      if (!collectedExceptions.isEmpty()) {
        return null;
      }
      try {
        return builder.build();
      } catch (Exception e) {
        throw new SchemaDoesntMatchException(
            "Could convert schema for " + schemaInformation.getFullName(), e);
      }
    } else {
      return messageFromMap(
          schemaInformation,
          descriptor,
          tableRow,
          ignoreUnknownValues,
          allowMissingRequiredFields,
          unknownFields,
          changeType,
          changeSequenceNum,
          collectedExceptions);
    }
  }

  static TableSchema tableSchemaFromDescriptor(Descriptor descriptor) {
    List<TableFieldSchema> tableFields =

View on GitHub (pinned to 12126d8942)