apache/beam · error · RuntimeException

Unexpected beam type " + fieldSchema

Error message

Unexpected beam type " + fieldSchema

What it means

scalarToProtoValue falls back to PRIMITIVE_ENCODERS keyed by the Avro primitive type when no logical type is present. If the Avro type has no registered primitive encoder, it throws RuntimeException("Unexpected beam type ..."). This means the field's Avro type is one the BigQuery Storage API conversion doesn't handle (e.g. BYTES/NULL/FIXED in some positions).

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/AvroGenericRecordToStorageApiProto.java:575

      long picoseconds = nanoAdjustment * PICOS_PER_NANO;
      return buildTimestampPicosMessage(
          Preconditions.checkNotNull(descriptor).getMessageType(), seconds, picoseconds);
    }
    LogicalType logicalType = LogicalTypes.fromSchema(type.getType());

    if (logicalType != null) {
      @Nullable
      BiFunction<LogicalType, Object, Object> logicalTypeEncoder =
          LOGICAL_TYPE_ENCODERS.get(logicalType.getName());
      if (logicalTypeEncoder == null) {
        throw new IllegalArgumentException("Unsupported logical type " + logicalType.getName());
      }
      return logicalTypeEncoder.apply(logicalType, value);
    } else {
      @Nullable Function<Object, Object> encoder = PRIMITIVE_ENCODERS.get(type.getType().getType());
      if (encoder == null) {
        throw new RuntimeException("Unexpected beam type " + fieldSchema);
      }
      return encoder.apply(value);
    }
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Change the field's Avro type to a supported primitive (STRING, LONG, DOUBLE, BOOLEAN, BYTES where supported)
2. Inspect the logged fieldSchema to identify the offending type and align your schema with the BigQuery column type
3. Convert FIXED to BYTES or STRING before writing
4. Upgrade Beam for newer type support

Example fix

// before
Schema fixed = Schema.createFixed("Md5", null, null, 16);
// after
Schema fixed = Schema.create(Schema.Type.BYTES);

// Java
Set<Schema.Type> supported = Set.of(STRING, LONG, DOUBLE, BOOLEAN, BYTES, INT);
if (!supported.contains(avroType.getType())) {
  throw new IllegalStateException("Unsupported Avro primitive: " + avroType.getType());
}

// Java
Function<Object, Object> enc = PRIMITIVE_ENCODERS.get(type.getType().getType());
if (enc != null) { /* encodable */ }

// Java
try {
  return scalarToProtoValue(fieldSchema, type, value);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Unexpected beam type")) {
    // convert the field to a supported primitive upstream
  }
  throw e;
}

When it happens

Trigger: Converting an Avro field whose primitive type isn't in PRIMITIVE_ENCODERS (e.g. FIXED, NULL-typed union branches resolved oddly) when writing GenericRecords to BigQuery Storage API

Common situations: Schemas using Avro FIXED types, fields typed NULL, or schema/type drift between data and declared table schema.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Does not support converting unknown type value: " + beamFiel
• is not primitive type.
• Reserved field name <field.name()> in user schema.
• Unsupported type <elementType.getType()>
• Received null value for non-nullable field " + fieldDescript

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4380bc9908f92315. Report an issue: GitHub.