apache/beam · error · RuntimeException

Unexpected null element type for the map's key on " +…

Error message

Unexpected null element type for the map's key on " + field.getName()

What it means

When converting a MAP-typed Beam field to a BigQuery STRUCT (BigQuery has no map type; maps become repeated structs of key/value), the converter reads the map's key type via FieldType.getMapKeyType(). A null key type means the FieldType is malformed or was constructed in a way that left the key type undefined, so no BigQuery mapping is possible.

Solutions

Build map fields with FieldType.map(keyType, valueType) so both key and value types are set.
Validate the schema up front: for every MAP field assert getMapKeyType() and getMapValueType() are non-null before calling the sink.
If the schema comes from another connector, re-declare it explicitly instead of reusing the inferred schema.
If null-ness is legitimately possible upstream, wrap conversion in try-catch and skip/fail the field with a clear message.

Example fix

// before
FieldType badMap = FieldType.builder().setType(TypeName.MAP).build();
// after
FieldType goodMap = FieldType.map(FieldType.string(), FieldType.int64());

Defensive patterns

Strategy: validation

Validate before calling

schema.getFields().stream()
  .filter(f -> f.getType().getTypeName() == Schema.TypeName.MAP)
  .forEach(f -> {
    if (f.getType().getMapKeyType() == null)
      throw new IllegalArgumentException("Map field " + f.getName() + " missing key type");
  });

Type guard

boolean hasCompleteMapType(Schema.Field f) {
  return f.getType().getTypeName() != Schema.TypeName.MAP
      || (f.getType().getMapKeyType() != null && f.getType().getMapValueType() != null);
}

Try / catch

try {
  protoTableSchemaFromBeamSchema(schema);
} catch (RuntimeException e) {
  if (e.getMessage().contains("null element type for the map's key")) {
    throw new SchemaException("Malformed map field in schema: " + e.getMessage());
  }
  throw e;
}

Prevention

Always use FieldType.map(k, v) factory methods
Avoid raw Map fields in @DefaultSchema record classes
Validate schemas in tests before running the pipeline

When it happens

Trigger: fieldDescriptorFromBeamField hitting case MAP where field.getType().getMapKeyType() returns null - i.e. a FieldType with TypeName.MAP that was not built via FieldType.map(keyType, valueType) or was mutated/deserialized incorrectly.

Common situations: Programmatically assembled schemas ( FieldType.builder().setType(TypeName.MAP) without a key type), schemas from custom format deserializers, or legacy serialized schemas where map metadata was lost.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ee6dc2b55f37822d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProto.java:276

            throw new RuntimeException(
                "Unsupported precision for Timestamp logical type " + precision);
          }
          // Map Timestamp.NANOS logical type to BigQuery TIMESTAMP(12) for nanosecond precision
          type = TableFieldSchema.Type.TIMESTAMP;
          builder.setTimestampPrecision(Int64Value.newBuilder().setValue(12L).build());
        } else {
          type = LOGICAL_TYPES.get(logicalType.getIdentifier());
          if (type == null) {
            throw new RuntimeException("Unsupported logical type " + field.getType());
          }
        }
        builder = builder.setType(type);
        break;
      case MAP:
        @Nullable FieldType keyType = field.getType().getMapKeyType();
        @Nullable FieldType valueType = field.getType().getMapValueType();
        if (keyType == null) {
          throw new RuntimeException(
              "Unexpected null element type for the map's key on " + field.getName());
        }
        if (valueType == null) {
          throw new RuntimeException(
              "Unexpected null element type for the map's value on " + field.getName());
        }

        builder =
            builder
                .setType(TableFieldSchema.Type.STRUCT)
                .addFields(fieldDescriptorFromBeamField(Field.of("key", keyType)))
                .addFields(fieldDescriptorFromBeamField(Field.of("value", valueType)))
                .setMode(TableFieldSchema.Mode.REPEATED);
        break;
      default:
        @Nullable
        TableFieldSchema.Type primitiveType = PRIMITIVE_TYPES.get(field.getType().getTypeName());
        if (primitiveType == null) {

View on GitHub (pinned to 12126d8942)