apache/beam · error · ValueError

Encountered an unsupported type: {field['type']!r}

Error message

Encountered an unsupported type: {field['type']!r}

What it means

Raised by generate_user_type_from_bq_schema when a field in the BigQuery table schema has a 'type' not present in effective_types (the set of BigQuery types Beam can map to a Python user type). Beam refuses to guess a mapping and fails fast rather than producing a wrong namedtuple schema.

Source

Thrown at sdks/python/apache_beam/io/gcp/bigquery_schema_tools.py:86

      BIG_QUERY_TO_PYTHON_TYPES. For example:
      ``{'DATE': datetime.date, 'JSON': dict}``

  Returns:
    type: type that can be used to work with pCollections.
  """
  effective_types = {**BIG_QUERY_TO_PYTHON_TYPES, **(type_overrides or {})}
  the_schema = beam.io.gcp.bigquery_tools.get_dict_table_schema(
      the_table_schema)
  if the_schema == {}:
    raise ValueError("Encountered an empty schema")
  field_names_and_types = []
  for field in the_schema['fields']:
    if selected_fields is not None and field['name'] not in selected_fields:
      continue
    if field['type'] in effective_types:
      typ = bq_field_to_type(field['type'], field['mode'], type_overrides)
    else:
      raise ValueError(
          f"Encountered "
          f"an unsupported type: {field['type']!r}")
    field_names_and_types.append((field['name'], typ))
  sample_schema = beam.typehints.schemas.named_fields_to_schema(
      field_names_and_types)
  usertype = beam.typehints.schemas.named_tuple_from_schema(sample_schema)
  return usertype


def bq_field_to_type(field, mode, type_overrides=None):
  """Convert a BigQuery field type and mode to a Python type hint.

  Args:
    field: The BigQuery type name (e.g., 'STRING', 'DATE').
    mode: The field mode ('NULLABLE', 'REPEATED', 'REQUIRED').
    type_overrides: Optional mapping of BigQuery type names (uppercase)
      to Python types. These override the default mappings.

View on GitHub (pinned to 12126d8942)

Solutions

1. Add a type_overrides entry mapping the offending field name to a concrete Python type, e.g. {'geo': bytes} or {'j': str}.
2. Upgrade apache-beam to a version whose effective_types includes the new BigQuery type (GEOGRAPHY/JSON support was added over time).
3. Fix typos in hand-authored schema JSON so field['type'] matches a valid BigQuery type name.
4. Pre-process the schema to drop or transform unsupported fields before calling convert_to_usertype.

Example fix

# before
convert_to_usertype(table_schema)  # schema has GEOGRAPHY field

# after
convert_to_usertype(table_schema, type_overrides={'location': bytes})

SUPPORTED = {'STRING','BYTES','INTEGER','INT64','FLOAT','FLOAT64','NUMERIC','BIGNUMERIC','BOOLEAN','BOOL','TIMESTAMP','DATE','TIME','DATETIME','RECORD','STRUCT'}
for f in schema['fields']:
    if f['type'] not in SUPPORTED and f['name'] not in (type_overrides or {}):
        raise ValueError(f"unsupported BQ type {f['type']!r} for field {f['name']!r}: add a type_overrides entry")

def is_supported_field(field, overrides=None):
    return field.get('type') in SUPPORTED_TYPES or (overrides or {}).get(field.get('name')) is not None

try:
    usertype = convert_to_usertype(schema, type_overrides=overrides)
except ValueError as e:
    log.error('schema conversion failed: %s', e)
    raise

When it happens

Trigger: Calling convert_to_usertype(table_schema) (directly or from WriteToBigQuery schema conversion) with a schema containing a field whose type is not one of the supported effective_types and for which no type_overrides entry maps it to a concrete Python type.

Common situations: Schemas using newer/less common BQ types like GEOGRAPHY, JSON, INTERVAL, or RANGE with older Beam versions; custom or hand-written JSON schemas with typos like 'STRINGG'; passing table schema fields like TIMESTAMP before overrides were configured.

Related errors

• Converting BigQuery type [{field_type}] to Python Beam type
• Unexpected schema argument: %s.
• Both a query and an output type of 'BEAM_ROW' were specified
• Unknown BigQuery field mode: {}
• Encountered an unsupported mode: {mode!r}

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e96b2de29a923e65. Report an issue: GitHub.