apache/beam · error · RuntimeException

Unknown logical type " + identifier

Error message

Unknown logical type " + identifier

What it means

Thrown as a RuntimeException when convertAvroFormat encounters a Beam LOGICAL_TYPE field whose identifier is not DATE, TIME, DATETIME, a known SQL date-time type, or a PassThroughLogicalType. Beam's BigQuery Avro converter simply has no mapping for that logical type identifier, so it refuses to convert. Usually indicates schema drift or a logical type added in a newer Beam than the running code.

Source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryUtils.java:1073

        } else if (SqlTypes.TIME.getIdentifier().equals(identifier)) {
          return convertAvroTime(avroValue);
        } else if (SqlTypes.DATETIME.getIdentifier().equals(identifier)) {
          return convertAvroDateTime(avroValue);
        } else if (SQL_DATE_TIME_TYPES.contains(identifier)) {
          switch (options.getTruncateTimestamps()) {
            case TRUNCATE:
              return truncateToMillis(avroValue);
            case REJECT:
              return safeToMillis(avroValue);
            default:
              throw new IllegalArgumentException(
                  String.format(
                      "Unknown timestamp truncation option: %s", options.getTruncateTimestamps()));
          }
        } else if (logicalType instanceof PassThroughLogicalType) {
          return convertAvroFormat(logicalType.getBaseType(), avroValue, options);
        } else {
          throw new RuntimeException("Unknown logical type " + identifier);
        }
      case ROW:
        Schema rowSchema = beamFieldType.getRowSchema();
        if (rowSchema == null) {
          throw new IllegalArgumentException("Nested ROW missing row schema");
        }
        GenericData.Record record = (GenericData.Record) avroValue;
        return toBeamRow(record, rowSchema, options);
      case MAP:
        return convertAvroRecordToMap(beamFieldType, avroValue, options);
      default:
        throw new RuntimeException(
            "Does not support converting unknown type value: " + beamFieldTypeName);
    }
  }

  private static ReadableInstant safeToMillis(Object value) {
    long subMilliPrecision = ((long) value) % 1000;

View on GitHub (pinned to 12126d8942)

Solutions

1. Upgrade the Beam SDK to the version that supports the logical type identifier in the error message.
2. Make the custom LogicalType extend PassThroughLogicalType so its base type is converted instead.
3. Change the field to a standard supported type (DATE/TIME/DATETIME or primitive) in the Beam schema.
4. Strip or remap unknown logical types from the schema before the BigQueryIO avro conversion.

Example fix

// before
public class MyType implements LogicalType<...> { /* custom identifier */ }
// after
public class MyType extends PassThroughLogicalType<String> { /* base type converted instead */ }

// Java: pre-check every logical type is supported before reading
for (Field f : schema.getFields()) {
  if (f.getType().getTypeName() == TypeName.LOGICAL_TYPE) {
    String id = f.getType().getLogical().getIdentifier();
    boolean supported = id.equals("DATE") || id.equals("TIME") || id.equals("DATETIME")
        || f.getType().getLogical() instanceof PassThroughLogicalType;
    if (!supported) throw new IllegalStateException("Unsupported logical type: " + id);
  }
}

// Java
static boolean isSupportedLogicalType(FieldType t) {
  return t.getTypeName() != TypeName.LOGICAL_TYPE
      || t.getLogical() instanceof PassThroughLogicalType
      || Set.of("DATE", "TIME", "DATETIME").contains(t.getLogical().getIdentifier());
}

try { rows = pipeline.apply(BigQueryIO.readTableRows()...); } catch (RuntimeException e) { if (e.getMessage().startsWith("Unknown logical type")) { log.error("Upgrade Beam or map logical type: {}", e.getMessage()); } throw e; }

When it happens

Trigger: BigQueryIO.readTableRows over Avro records whose Beam schema field has a custom/unrecognized logical type identifier — e.g. a schema produced by a newer Beam version with logical types this converter doesn't know, or user-defined logical types on ROW fields.

Common situations: Beam version skew (newer producer schema, older consumer); user-defined LogicalType subclasses that don't extend PassThroughLogicalType; schemas round-tripped through systems adding custom logical types.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• Unknown logical type
• Logical types don't match and cannot be merged: +identifier1
• Local timestamp (micros) can only be used with an underlying
• Reserved field name <field.name()> in user schema.
• Unsupported logical type " + logicalType.getName()

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0cfb72b9ded448dd. Report an issue: GitHub.