What causes "Unsupported atomic type: {0}" in apache/beam?

Use a Beam atomic type with a direct pyarrow equivalent (string, int64, double, etc.)

apache/beam · error · ValueError

Unsupported atomic type

Error message

Unsupported atomic type: {0}

What it means

_arrow_type_from_beam_fieldtype maps Beam atomic types to pyarrow types via the ATOMIC_TYPE_TO_PYARROW table. If the Beam FieldType's atomic_type has no pyarrow equivalent in the table, it raises ValueError 'Unsupported atomic type'.

Solutions

Use a Beam atomic type with a direct pyarrow equivalent (string, int64, double, etc.)
Upgrade apache-beam so the mapping table includes your type
Pre-convert the field to a supported type in your pipeline (e.g. cast to int64/string) before schema conversion

Example fix

// before
beam.Row(x=beam_logical_type_value)  # atomic type not in ATOMIC_TYPE_TO_PYARROW
// after
beam.Row(x=int(beam_logical_type_value))  # plain int64, supported

Defensive patterns

Strategy: type-guard

Validate before calling

import apache_beam.typehints.schema as schema_pb2
from apache_beam.typehints.arrow_type_compatibility import ATOMIC_TYPE_TO_PYARROW
if ft.WhichOneof('type_info') == 'atomic_type' and ft.atomic_type not in ATOMIC_TYPE_TO_PYARROW:
    raise TypeError(f'atomic type {ft.atomic_type} has no pyarrow mapping; use a primitive type')

Type guard

def arrow_supported(ft) -> bool:
    from apache_beam.typehints.arrow_type_compatibility import ATOMIC_TYPE_TO_PYARROW
    return ft.WhichOneof('type_info') != 'atomic_type' or ft.atomic_type in ATOMIC_TYPE_TO_PYARROW

Try / catch

try:
    arrow_type = _arrow_type_from_beam_fieldtype(ft)
except ValueError as e:
    if 'Unsupported atomic type' in str(e):
        ft = cast_to_supported_primitive(ft)
        arrow_type = _arrow_type_from_beam_fieldtype(ft)
    else:
        raise

Prevention

Restrict row types to primitive fields when planning pyarrow batching
Convert custom/logical atomic types to base primitives upstream
Keep apache-beam updated so new atomic types are mapped

When it happens

Trigger: Converting a Beam schema containing an atomic type absent from the mapping (e.g. Beam logical/bytes-backed specializations or proto-defined atomic types not in ATOMIC_TYPE_TO_PYARROW) through _arrow_field_from_beam_fieldtype, _make_arrow_map, or PyarrowBatchConverter construction.

Common situations: Using Beam logical types (e.g. its sql/LogicalType wrappers) with arrow batch conversion; schemas exchanged from other Beam SDKs (Java/Go) with exotic atomic types; adding new Beam atomic types while on an old Beam/pyarrow version.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7d6be07b0e24f0f0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/typehints/arrow_type_compatibility.py:272

        _arrow_field_from_beam_fieldtype(beam_map_type.value_type))

  def _arrow_map_to_beam_map(arrow_map_type):
    return schema_pb2.MapType(
        key_type=_beam_fieldtype_from_arrow_field(arrow_map_type.key_field),
        value_type=_beam_fieldtype_from_arrow_field(arrow_map_type.item_field))


def _arrow_type_from_beam_fieldtype(
    beam_fieldtype: schema_pb2.FieldType,
) -> Tuple[pa.DataType, Optional[Dict[bytes, bytes]]]:
  # Note this function is not concerned with beam_fieldtype.nullable, as
  # nullability is a property of the Field in Arrow.
  type_info = beam_fieldtype.WhichOneof("type_info")
  if type_info == 'atomic_type':
    try:
      output_arrow_type = ATOMIC_TYPE_TO_PYARROW[beam_fieldtype.atomic_type]
    except KeyError:
      raise ValueError(
          "Unsupported atomic type: {0}".format(beam_fieldtype.atomic_type))
  elif type_info == "array_type":
    output_arrow_type = pa.list_(
        _arrow_field_from_beam_fieldtype(
            beam_fieldtype.array_type.element_type))
  elif type_info == "map_type":
    output_arrow_type = _make_arrow_map(beam_fieldtype.map_type)
  elif type_info == "row_type":
    schema = beam_fieldtype.row_type.schema
    # Note schema id and options are handled at the arrow field level, they are
    # added at field-level metadata.
    output_arrow_type = pa.struct(
        [_arrow_field_from_beam_field(field) for field in schema.fields])
  elif type_info == "logical_type":
    # TODO(https://github.com/apache/beam/issues/23817): Add support for logical
    # types.
    raise NotImplementedError(
        "Beam logical types are not currently supported "

View on GitHub (pinned to 12126d8942)