apache/beam · error · NotImplementedError

Beam logical types are not currently supported in…

Error message

Beam logical types are not currently supported in arrow_type_compatibility.

What it means

Beam logical types (schema_pb2 logical_type) are not implemented in the arrow conversion layer (tracked as beam issue 23817). When _arrow_type_from_beam_fieldtype encounters type_info == 'logical_type' it raises NotImplementedError.

Solutions

Replace the logical type field with its underlying representation (e.g. use datetime/int64/string base types)
Convert logical values to primitive types before the arrow batch conversion
Track/upgrade for Beam support of issue 23817 (logical type support in arrow conversion)

Example fix

// before
beam.Row(ts=LogicalType('sql:DATE', value))
// after
beam.Row(ts=datetime.date(value))  # plain date, arrow-supported

Defensive patterns

Strategy: fallback

Validate before calling

if ft.WhichOneof('type_info') == 'logical_type':
    raise TypeError('logical types not supported for arrow conversion; flatten to primitives first')

Type guard

def is_logical_type(ft) -> bool:
    return ft.WhichOneof('type_info') == 'logical_type'

Try / catch

try:
    arrow_type = _arrow_type_from_beam_fieldtype(ft)
except NotImplementedError:
    ft = to_underlying_primitive(ft)
    arrow_type = _arrow_type_from_beam_fieldtype(ft)

Prevention

Avoid Beam logical-type fields in PCollections destined for arrow batching
Represent logical values as base types (datetime, int, str)
Track beam issue 23817 for when support lands

When it happens

Trigger: Converting a Beam schema containing a logical-type field (e.g. types registered via beam.typehints with logical type constructors, or SqlType wrappers) to arrow via PyarrowBatchConverter or _arrow_field_from_beam_fieldtype.

Common situations: Using Beam SQL logical types (DATE, TIME, DECIMAL via zetasql wrappers) in a PCollection then batching to pa.Table; cross-SDK schemas carrying logical types; third-party logical type registrations.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9f96b4ba90b4e917. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/typehints/arrow_type_compatibility.py:289

    except KeyError:
      raise ValueError(
          "Unsupported atomic type: {0}".format(beam_fieldtype.atomic_type))
  elif type_info == "array_type":
    output_arrow_type = pa.list_(
        _arrow_field_from_beam_fieldtype(
            beam_fieldtype.array_type.element_type))
  elif type_info == "map_type":
    output_arrow_type = _make_arrow_map(beam_fieldtype.map_type)
  elif type_info == "row_type":
    schema = beam_fieldtype.row_type.schema
    # Note schema id and options are handled at the arrow field level, they are
    # added at field-level metadata.
    output_arrow_type = pa.struct(
        [_arrow_field_from_beam_field(field) for field in schema.fields])
  elif type_info == "logical_type":
    # TODO(https://github.com/apache/beam/issues/23817): Add support for logical
    # types.
    raise NotImplementedError(
        "Beam logical types are not currently supported "
        "in arrow_type_compatibility.")
  else:
    raise ValueError(f"Unrecognized type_info: {type_info!r}")

  return output_arrow_type


class PyarrowBatchConverter(BatchConverter):
  def __init__(self, element_type: RowTypeConstraint):
    super().__init__(pa.Table, element_type)
    self._beam_schema = typing_to_runner_api(element_type).row_type.schema
    arrow_schema = arrow_schema_from_beam_schema(self._beam_schema)

    self._arrow_schema = arrow_schema

  @staticmethod
  def from_typehints(element_type,

View on GitHub (pinned to 12126d8942)