apache/beam · error · RuntimeError

Unexpected field type

Error message

Unexpected field type: %s

What it means

Raised by `_convert_cell_value_to_dict` (called from `convert_row_to_dict`) when a BigQuery field type is not among the handled types (STRING, INTEGER, FLOAT, BOOLEAN, TIMESTAMP, RECORD, NUMERIC, GEOGRAPHY, etc.). Beam encountered a table schema field type its row-to-dict converter doesn't implement, so it aborts with a RuntimeError naming the type.

Solutions

  1. Upgrade apache-beam to the latest version, which handles newer BigQuery field types.
  2. Cast unsupported columns in your SQL (e.g. CAST(json_col AS STRING)) so the schema Beam sees only contains supported types.
  3. Restrict the read schema (SELECT specific columns) to exclude unhandled fields.
  4. If stuck on an old Beam version, convert the offending column's type in BigQuery (ALTER TABLE / materialized view) to a supported equivalent like STRING.

Example fix

// before
SELECT * FROM `project.dataset.table_with_json_column`
// after
SELECT id, CAST(json_col AS STRING) AS json_col, ts FROM `project.dataset.table_with_json_column`
Defensive patterns

Strategy: try-catch

Validate before calling

SUPPORTED = {'STRING','INTEGER','FLOAT','BOOLEAN','TIMESTAMP','RECORD','NUMERIC','GEOGRAPHY','DATE','TIME','DATETIME','BYTES'}
def schema_types_supported(table_schema):
    return all(f.type in SUPPORTED for f in table_schema.fields)
# check before reading rows

Try / catch

try:
    row_dict = convert_row_to_dict(row, schema)
except RuntimeError as e:
    if str(e).startswith('Unexpected field type'):
        log.warning('skipping row with unsupported field type: %s', e)
        return None
    raise

Prevention

When it happens

Trigger: Reading/queried rows from a table whose schema includes a newer or unhandled type (e.g. JSON, RANGE, INTERVAL, or other post-support types) via convert_row_to_dict / BigQuery read paths.

Common situations: Tables created with recently added BigQuery types used with an older apache-beam version; mixing tools (someone altered the table schema after the pipeline was written); using Beam to read a table written by other tools with exotic types.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/141884d9946b926d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/gcp/bigquery_tools.py:1383

      # Input: "2016-11-03" --> Output: "2016-11-03"
      return value
    elif field.type == 'DATETIME':
      # Input: "2016-11-03T00:49:36" --> Output: "2016-11-03T00:49:36"
      return value
    elif field.type == 'TIME':
      # Input: "00:49:36" --> Output: "00:49:36"
      return value
    elif field.type == 'RECORD':
      # Note that a schema field object supports also a RECORD type. However
      # when querying, the repeated and/or record fields are flattened
      # unless we pass the flatten_results flag as False to the source
      return self.convert_row_to_dict(value, field)
    elif field.type == 'NUMERIC':
      return decimal.Decimal(value)
    elif field.type == 'GEOGRAPHY':
      return value
    else:
      raise RuntimeError('Unexpected field type: %s' % field.type)

  def convert_row_to_dict(self, row, schema):
    """Converts a TableRow instance using the schema to a Python dict."""
    result = {}
    for index, field in enumerate(schema.fields):
      value = None
      if isinstance(schema, bigquery.TableSchema):
        cell = row.f[index]
        value = from_json_value(cell.v) if cell.v is not None else None
      elif isinstance(schema, bigquery.TableFieldSchema):
        cell = row['f'][index]
        value = cell['v'] if 'v' in cell else None
      if field.mode == 'REPEATED':
        if value is None:
          # Ideally this should never happen as repeated fields default to
          # returning an empty list
          result[field.name] = []
        else:

View on GitHub (pinned to 12126d8942)