apache/beam · error · RuntimeException

Unexpected beam type " + beamFieldType

Error message

Unexpected beam type " + beamFieldType

What it means

scalarToProtoValue maps each primitive Beam FieldType to a protobuf encoder via the PRIMITIVE_ENCODERS map. When the field's type name has no registered encoder, it throws this RuntimeException, meaning the Beam type cannot be represented as a scalar in the BigQuery Storage API proto schema.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BeamRowToStorageApiProto.java:408

      }
      if (logicalType.getIdentifier().equals(Timestamp.IDENTIFIER)) {
        Instant instant = (Instant) value;
        Descriptor timestampPicosDescriptor =
            Preconditions.checkNotNull(fieldDescriptor).getMessageType();
        return buildTimestampPicosMessage(timestampPicosDescriptor, instant);
      }
      @Nullable
      BiFunction<LogicalType<?, ?>, Object, Object> logicalTypeEncoder =
          LOGICAL_TYPE_ENCODERS.get(logicalType.getIdentifier());
      if (logicalTypeEncoder == null) {
        throw new RuntimeException("Unsupported logical type " + logicalType.getIdentifier());
      }
      return logicalTypeEncoder.apply(logicalType, value);
    } else {
      @Nullable
      Function<Object, Object> encoder = PRIMITIVE_ENCODERS.get(beamFieldType.getTypeName());
      if (encoder == null) {
        throw new RuntimeException("Unexpected beam type " + beamFieldType);
      }
      return encoder.apply(value);
    }
  }

  static Object mapEntryToProtoValue(
      Descriptor descriptor,
      FieldType keyFieldType,
      FieldType valueFieldType,
      Map.Entry<Object, Object> entryValue) {
    DynamicMessage.Builder builder = DynamicMessage.newBuilder(descriptor);
    FieldDescriptor keyFieldDescriptor =
        Preconditions.checkNotNull(descriptor.findFieldByName("key"));
    @Nullable Object key = toProtoValue(keyFieldDescriptor, keyFieldType, entryValue.getKey());
    if (key != null) {
      builder.setField(keyFieldDescriptor, key);
    }
    FieldDescriptor valueFieldDescriptor =

View on GitHub (pinned to 12126d8942)

Solutions

1. Change the field to a supported Beam type (BOOLEAN, INT64, FLOAT, DOUBLE, STRING, BYTES, DATETIME, logical types)
2. Pre-convert unsupported types to strings or supported primitives before the sink
3. Check the Beam version for known gaps between Schema types and BigQuery support and upgrade
4. Use a different BigQuery sink method that tolerates the type

Example fix

// before
Schema.Field.of("blob", Schema.FieldType.iterable(Schema.FieldType.BYTE))
// after
Schema.Field.of("blob", Schema.FieldType.BYTES)

Set<Schema.TypeName> supported = Set.of(BOOLEAN, INT64, FLOAT, DOUBLE, STRING, BYTES, DATETIME);
if (!supported.contains(field.getType().getTypeName())) throw new IllegalArgumentException("Type " + field.getType() + " not supported by BigQuery Storage API sink");

When it happens

Trigger: Writing a Row containing a primitive Beam FieldType that the BigQuery Storage API sink has no encoder for (e.g. unsupported nested/byte type used as a scalar) to a BigQuery sink using the Storage Write API.

Common situations: Schemas built programmatically with an exotic FieldType (e.g. unsupported iterable/row nested as a scalar slot), or after Beam version changes where a type is added to Schema but not yet mapped to BigQuery NUMERIC/JSON/proto encoders.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Unsupported logical type " + logicalType.getIdentifier()
• Cannot convert BigQuery type '' to '' because the BigQuery t
• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b584abb6dd55e913. Report an issue: GitHub.