apache/beam · error · IllegalArgumentException

Error processing struct to row: {e.getMessage()}

Error message

Error processing struct to row: {e.getMessage()}

What it means

structTypeToBeamRowSchema wraps an IllegalArgumentException raised while converting a Spanner struct field's type into a Beam Schema.FieldType, rethrowing it with the prefix "Error processing struct to row: ". It indicates at least one column of the Spanner STRUCT/row has a type the converter does not support; the original message is appended.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/StructUtils.java:81

        schema.getFields().stream()
            .collect(
                HashMap::new,
                (map, field) -> map.put(field.getName(), getStructValue(struct, field)),
                Map::putAll);
    return Row.withSchema(schema).withFieldValues(structValues).build();
  }

  public static Schema structTypeToBeamRowSchema(StructType structType, boolean isRead) {
    Schema.Builder beamSchema = Schema.builder();
    structType
        .getFieldsList()
        .forEach(
            field -> {
              Schema.FieldType fieldType;
              try {
                fieldType = convertSpannerTypeToBeamFieldType(field.getType());
              } catch (IllegalArgumentException e) {
                throw new IllegalArgumentException(
                    "Error processing struct to row: " + e.getMessage());
              }
              // Treat reads from Spanner as Nullable and leave Null handling to Spanner
              if (isRead) {
                beamSchema.addNullableField(field.getName(), fieldType);
              } else {
                beamSchema.addField(field.getName(), fieldType);
              }
            });
    return beamSchema.build();
  }

  public static Schema.FieldType convertSpannerTypeToBeamFieldType(
      com.google.spanner.v1.Type spannerType) {
    switch (spannerType.getCode()) {
      case BOOL:
        return Schema.FieldType.BOOLEAN;
      case BYTES:

View on GitHub (pinned to 12126d8942)

Solutions

  1. Read the wrapped message to identify the offending field type and remove/flatten unsupported columns in the query (e.g. avoid selecting STRUCT columns).
  2. Upgrade the Beam Google Cloud Platform SDK to a version that maps the offending Spanner type.
  3. Cast unsupported columns to supported types (STRING, NUMERIC, etc.) in the SQL query before reading.

Example fix

// before
SELECT id, nested_struct FROM my_table
// after
SELECT id, TO_JSON_STRING(nested_struct) AS nested_struct_json FROM my_table
Defensive patterns

Strategy: try-catch

Validate before calling

// pre-check schema fields against known-supported Spanner codes
Set<String> supported = Set.of("BOOL","INT64","FLOAT64","NUMERIC","STRING","DATE","TIMESTAMP","BYTES","ARRAY");
boolean allSupported = type.getStructFields().stream()
    .allMatch(f -> supported.contains(f.getType().getCode().name()));

Try / catch

try {
  Schema schema = StructUtils.structTypeToBeamRowSchema(spannerType, isRead);
} catch (IllegalArgumentException e) {
  LOG.error("Unsupported Spanner schema: {}", e.getMessage());
  throw new PipelineException("Failing read; see cause for the unsupported column type", e);
}

Prevention

When it happens

Trigger: Calling StructUtils.structTypeToBeamRowSchema (directly or via SpannerIO reads of STRUCT columns) on a Spanner type whose code hits an unsupported branch in convertSpannerTypeToBeamFieldType, e.g. a STRUCT-typed or otherwise unrecognized field type.

Common situations: Reading Spanner tables containing STRUCT columns or newer Spanner types (e.g. proto/json columns on newer SDK/backends) with an older Beam SDK that lacks the mapping.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/fd557d7714f4316c. Report an issue: GitHub.