apache/beam · error · ValueError

Converting BigQuery type [{field_type}] to Python Beam type

Error message

Converting BigQuery type [{field_type}] to Python Beam type is not supported.

What it means

get_beam_typehints_from_tableschema() maps BigQuery field types to Beam Python type hints and raises ValueError when a field's type has no mapping in effective_types (after applying type_overrides). Unsupported or unrecognized BigQuery/legacy type names cannot be converted to Beam Rows type hints.

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery_tools.py:1822

  Returns:
    List[Tuple[str, Any]]: A list of type hints that describe the input schema.
    Nested and repeated fields are supported.
  """
  effective_types = {**BIGQUERY_TYPE_TO_PYTHON_TYPE, **(type_overrides or {})}
  if not isinstance(schema, (bigquery.TableSchema, bigquery.TableFieldSchema)):
    schema = get_bq_tableschema(schema)
  typehints = []
  for field in schema.fields:
    name, field_type, mode = field.name, field.type.upper(), field.mode.upper()

    if field_type in ["STRUCT", "RECORD"]:
      # Structs can be represented as Beam Rows.
      typehint = RowTypeConstraint.from_fields(
          get_beam_typehints_from_tableschema(field, type_overrides))
    elif field_type in effective_types:
      typehint = effective_types[field_type]
    else:
      raise ValueError(
          f"Converting BigQuery type [{field_type}] to "
          "Python Beam type is not supported.")

    if mode == "REPEATED":
      typehint = Sequence[typehint]
    elif mode != "REQUIRED":
      typehint = Optional[typehint]

    typehints.append((name, typehint))
  return typehints


class BigQueryJobTypes:
  EXPORT = 'EXPORT'
  COPY = 'COPY'
  LOAD = 'LOAD'
  QUERY = 'QUERY'

View on GitHub (pinned to 12126d8942)

Solutions

Remove or exclude the unsupported column from the schema before reading/writing.
Supply type_overrides mapping the unsupported BigQuery type to a supported one (e.g. {'GEOGRAPHY': 'STRING'}).
Cast the column in SQL (e.g. CAST(geog AS STRING)) in the query used to export data.
Upgrade apache-beam to a version that maps the newer BigQuery types.

Example fix

// before
WriteToBigQuery(schema=table_schema)  # schema has GEOGRAPHY field

// after
WriteToBigQuery(
    schema=table_schema,
    type_overrides={"GEOGRAPHY": "STRING"})

Defensive patterns

Strategy: validation

Validate before calling

SUPPORTED = {'STRING','BYTES','INTEGER','INT64','FLOAT','FLOAT64','NUMERIC','BOOLEAN','BOOL','TIMESTAMP','DATE','TIME','DATETIME','RECORD','STRUCT'}
unsupported = [f['type'] for f in schema['fields'] if f['type'].strip().upper() not in SUPPORTED and f['type'] not in (type_overrides or {})]
if unsupported:
    raise ValueError(f'Unsupported BigQuery types: {unsupported}')

Type guard

def is_convertible_field(field, overrides=None):
    t = field.get('type', '').strip().upper()
    return t in overrides or t in {'STRING','INTEGER','FLOAT','BOOLEAN','TIMESTAMP','RECORD','BYTES','NUMERIC','DATE','TIME','DATETIME'}

Try / catch

try:
    hints = get_beam_typehints_from_tableschema(schema)
except ValueError as e:
    if 'not supported' in str(e):
        hints = get_beam_typehints_from_tableschema(schema, type_overrides={'GEOGRAPHY': 'STRING', 'JSON': 'STRING'})
    else:
        raise

Prevention

Audit table schemas for GEOGRAPHY/JSON/other exotic types before using Rows-based reads.
Set type_overrides proactively for unsupported types.
Strip whitespace and normalize case in schema type names.
Keep apache-beam updated for newer BigQuery type mappings.

When it happens

Trigger: A table schema containing a BigQuery type not in the supported map (e.g. GEOGRAPHY, JSON, or a typo like 'STRING ' with whitespace or 'INTERVAL'); custom type_overrides that fail to cover an otherwise unsupported type; recursive struct fields containing an unsupported nested type.

Common situations: Reading modern BigQuery tables (GEOGRAPHY/JSON columns) into Beam Rows; schemas autogenerated from newer BigQuery features; hand-written schema strings with typos; older Beam versions lacking newer type mappings.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e25d5f84f25009ed. Report an issue: GitHub.