apache/beam · error · SchemaDoesntMatchException

Unexpected value: , type: . Table field name: , type

Error message

Unexpected value: ${value}, type: ${value.getClass()}. Table field name: ${schemaInformation.getFullName()}, type: ${schemaInformation.getType()}

What it means

After applying the type converter, the converted value came back null (or the input was null for a non-nullable field), so a SchemaDoesntMatchException is thrown describing the value, its Java class, and the table field's full name and declared type. It signals the value did not match what the declared BigQuery type can hold.

Solutions

Inspect the message's field name and type, then fix the supplied value's type/format at the source.
Pre-validate rows against the schema (types, nullability) before the sink.
Use null-safe casting in the producing transform (e.g. parse with fallback or dead-letter invalid rows).
Make nullable fields NULLABLE in the table schema if legitimate nulls must pass.

Example fix

// before
row.set("qty", "12a"); // INT64
// after
row.set("qty", Long.parseLong("12")); // or fix upstream data

Defensive patterns

Strategy: validation

Validate before calling

if (value == null && !"NULLABLE".equals(fieldMode)) throw new IllegalArgumentException(fieldFullName + " is not nullable"); if (value != null && converterWouldReturnNull(value, fieldType)) throw new IllegalArgumentException("bad value " + value + " for " + fieldType);

Type guard

boolean convertible(Object v, String bqType) { if (v == null) return "NULLABLE".equals(bqTypeMode); try { return convert(bqType, v) != null; } catch (Exception e) { return false; } }

Try / catch

try { messageFromTableRow(...); } catch (SchemaDoesntMatchException e) { LOG.warn("Bad value for field {}: {}", e.getMessage()); deadLetter(row); }

Prevention

Reject or repair stringly-typed values at ingest (parse/validate before the sink)
Route non-conforming rows to a dead-letter PCollection with the SchemaDoesntMatchException message
Declare legitimately-null fields NULLABLE in the table schema
Standardize value formats (ISO-8601, plain decimal strings) across producers

When it happens

Trigger: A converter (numeric parse, timestamp parse, bool parse, etc.) returns null because the supplied value's runtime type/format doesn't fit the declared field type — e.g. passing "abc" for INT64 or a Map for a FLOAT64 field.

Common situations: Stringly-typed ETL output feeding typed columns; nulls landing in REQUIRED fields; mixed-type JSON columns; locale-formatted numbers that fail parsing.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ec6e0ddd1eff22be. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java:1686

      }

      Descriptor messageType = Preconditions.checkArgumentNotNull(fieldDescriptor).getMessageType();
      converted =
          DynamicMessage.newBuilder(messageType)
              .setField(messageType.findFieldByName("seconds"), seconds)
              .setField(messageType.findFieldByName("picoseconds"), picoseconds)
              .build();

    } else {
      @Nullable ThrowingBiFunction<String, Object, @Nullable Object> converter =
          TYPE_MAP_PROTO_CONVERTERS.get(schemaInformation.getType());
      if (converter == null) {
        throw new RuntimeException("Unknown type " + schemaInformation.getType());
      }
      converted = converter.apply(schemaInformation.getFullName(), value);
    }
    if (converted == null) {
      throw new SchemaDoesntMatchException(
          "Unexpected value: "
              + value
              + ", type: "
              + (value == null ? "null" : value.getClass())
              + ". Table field name: "
              + schemaInformation.getFullName()
              + ", type: "
              + schemaInformation.getType());
    }
    return converted;
  }

  private static long toEpochMicros(Instant timestamp) {
    // i.e 1970-01-01T00:01:01.000040Z: 61 * 1000_000L + 40000/1000 = 61000040
    return timestamp.getEpochSecond() * 1000_000L + timestamp.getNano() / 1000;
  }

  @VisibleForTesting

View on GitHub (pinned to 12126d8942)