apache/beam · error · UnsupportedOperationException

Converting BigQuery type

Error message

Converting BigQuery type %s to Beam type is unsupported

What it means

When building a protobuf FieldDescriptorProto from a BigQuery TableFieldSchema, BigQuery types with no corresponding proto primitive type in PRIMITIVE_TYPES_BQ_TO_PROTO are rejected with an UnsupportedOperationException. It means the table declares a BigQuery type this converter cannot map (e.g. GEOGRAPHY, JSON, INTERVAL, RANGE depending on Beam version).

Solutions

Remove or cast the unsupported column (e.g. store JSON as STRING, GEOGRAPHY as STRING/WKT).
Upgrade Apache Beam — newer versions map more BigQuery types.
Exclude unsupported fields via select()/projection before writing.
Use a different sink path (e.g. FILE_LOADS) that supports the type.

Example fix

// before
schema field of type GEOGRAPHY fed to proto conversion
// after
schema field of type STRING holding WKT: SELECT ST_AsText(geo) AS geo

Defensive patterns

Strategy: validation

Validate before calling

Set<String> SUPPORTED = Set.of("BOOL","INT64","FLOAT64","NUMERIC","STRING","BYTES","TIMESTAMP","DATE","TIME","DATETIME"); List<String> unsupported = schema.getFieldsList().stream().map(TableFieldSchema::getType).filter(t -> !SUPPORTED.contains(t.name())).map(Enum::name).toList();

Type guard

boolean bqTypeSupported(TableFieldSchema.Type t) { return PRIMITIVE_TYPES_BQ_TO_PROTO.containsKey(t); }

Try / catch

try { protoFieldFor(schemaField); } catch (UnsupportedOperationException e) { LOG.warn("Skipping unsupported BQ type {}", schemaField.getType()); }

Prevention

Inspect the target table schema for GEOGRAPHY/JSON/INTERVAL/RANGE before using Storage API writes
Cast exotic types to STRING upstream
Upgrade Beam when new BigQuery types ship
Use FILE_LOADS sink for tables with unsupported column types

When it happens

Trigger: Writing to or reading from a BigQuery table whose schema contains newer BQ types (GEOGRAPHY, JSON, INTERVAL/RANGE) through the Storage Write/Avro-unsupported path that uses proto conversion.

Common situations: Target table created with JSON or GEOGRAPHY columns while the pipeline schema still declares them; BigQuery adding new types not yet mapped in the installed Beam version.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/4aa97d3adb815870. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/TableRowToStorageApiProto.java:1378

          if (!typeAlreadyExists) {
            descriptorBuilder.addNestedType(TIMESTAMP_PICOS_DESCRIPTOR_PROTO);
          }
          fieldDescriptorBuilder =
              fieldDescriptorBuilder
                  .setType(FieldDescriptorProto.Type.TYPE_MESSAGE)
                  .setTypeName(TIMESTAMP_PICOS_DESCRIPTOR_PROTO.getName());
        } else {
          // Microsecond precision - use simple INT64
          fieldDescriptorBuilder =
              fieldDescriptorBuilder.setType(FieldDescriptorProto.Type.TYPE_INT64);
        }
        break;

      default:
        FieldDescriptorProto.@Nullable Type type =
            PRIMITIVE_TYPES_BQ_TO_PROTO.get(fieldSchema.getType());
        if (type == null) {
          throw new UnsupportedOperationException(
              "Converting BigQuery type " + fieldSchema.getType() + " to Beam type is unsupported");
        }
        fieldDescriptorBuilder = fieldDescriptorBuilder.setType(type);
    }

    if (fieldSchema.getMode() == TableFieldSchema.Mode.REPEATED) {
      fieldDescriptorBuilder = fieldDescriptorBuilder.setLabel(Label.LABEL_REPEATED);
    } else if (!respectRequired || fieldSchema.getMode() != TableFieldSchema.Mode.REQUIRED) {
      fieldDescriptorBuilder = fieldDescriptorBuilder.setLabel(Label.LABEL_OPTIONAL);
    } else {
      fieldDescriptorBuilder = fieldDescriptorBuilder.setLabel(Label.LABEL_REQUIRED);
    }
    descriptorBuilder.addField(fieldDescriptorBuilder.build());
  }

  /**
   * mergeNewFields(original, newFields) unlike proto merge or concatenating proto bytes is merging
   * the main differences is skipping primitive fields that are already set and merging structs and

View on GitHub (pinned to 12126d8942)