apache/beam · error · IllegalArgumentException

Nested ROW missing row schema

Error message

Nested ROW missing row schema

What it means

Thrown as IllegalArgumentException when convertAvroFormat handles a Beam ROW-typed field whose FieldType has no row schema attached. To recursively convert a nested Avro record into a Beam Row, the field type must carry its nested Schema; a missing schema is treated as an invalid/incomplete schema definition. This indicates the Beam schema was built incorrectly rather than bad data.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java:1078

          switch (options.getTruncateTimestamps()) {
            case TRUNCATE:
              return truncateToMillis(avroValue);
            case REJECT:
              return safeToMillis(avroValue);
            default:
              throw new IllegalArgumentException(
                  String.format(
                      "Unknown timestamp truncation option: %s", options.getTruncateTimestamps()));
          }
        } else if (logicalType instanceof PassThroughLogicalType) {
          return convertAvroFormat(logicalType.getBaseType(), avroValue, options);
        } else {
          throw new RuntimeException("Unknown logical type " + identifier);
        }
      case ROW:
        Schema rowSchema = beamFieldType.getRowSchema();
        if (rowSchema == null) {
          throw new IllegalArgumentException("Nested ROW missing row schema");
        }
        GenericData.Record record = (GenericData.Record) avroValue;
        return toBeamRow(record, rowSchema, options);
      case MAP:
        return convertAvroRecordToMap(beamFieldType, avroValue, options);
      default:
        throw new RuntimeException(
            "Does not support converting unknown type value: " + beamFieldTypeName);
    }
  }

  private static ReadableInstant safeToMillis(Object value) {
    long subMilliPrecision = ((long) value) % 1000;
    if (subMilliPrecision != 0) {
      throw new IllegalArgumentException(
          String.format(
              "BigQuery data contained value %s with sub-millisecond precision, which Beam does"
                  + " not currently support."

View on GitHub (pinned to 12126d8942)

Solutions

  1. Attach the nested schema: use FieldType.row(Schema) or .withRowSchema(nestedSchema) when building the field type.
  2. Verify the top-level Schema was built with Schema.builder() so nested ROW fields get their schemas populated.
  3. If schemas cross a serialization boundary, ensure the row schema field is preserved (it is part of FieldType serialization).
  4. Log/inspect beamFieldType with Schema coder display to find which nested field lacks its schema.

Example fix

// before
FieldType nested = FieldType.row(); // no schema
// after
FieldType nested = FieldType.row(nestedSchema);
Defensive patterns

Strategy: validation

Validate before calling

// Java: verify every ROW field has a row schema before conversion
for (Field f : schema.getFields()) {
  if (f.getType().getTypeName() == TypeName.ROW && f.getType().getRowSchema() == null) {
    throw new IllegalStateException("Field " + f.getName() + " is ROW but has no row schema");
  }
}

Type guard

// Java
static boolean hasRowSchema(FieldType t) {
  return t.getTypeName() != TypeName.ROW || t.getRowSchema() != null;
}

Try / catch

try { return convertAvroFormat(field, value, opts); } catch (IllegalArgumentException e) { if ("Nested ROW missing row schema".equals(e.getMessage())) { throw new SchemaConfigurationException(field.getName()); } throw e; }

Prevention

When it happens

Trigger: A Beam schema field of type ROW was created via FieldType.row() without .getRowSchema()/withRowSchema, or the row schema was lost during schema serialization/deserialization, then BigQueryIO avro conversion recursed into that field.

Common situations: Hand-constructed FieldType.row() calls; schemas built dynamically/reflection-based where nested types weren't registered; schema transport through formats that drop nested schema metadata.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/31290471b9e56b04. Report an issue: GitHub.