{"record":{"id":"9f96b4ba90b4e917","repo":"apache/beam","slug":"beam-logical-types-are-not-currently-supported-in-arrow-type","errorCode":null,"errorMessage":"Beam logical types are not currently supported in arrow_type_compatibility.","messagePattern":"Beam logical types are not currently supported in arrow_type_compatibility\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/typehints/arrow_type_compatibility.py","lineNumber":289,"sourceCode":"    except KeyError:\n      raise ValueError(\n          \"Unsupported atomic type: {0}\".format(beam_fieldtype.atomic_type))\n  elif type_info == \"array_type\":\n    output_arrow_type = pa.list_(\n        _arrow_field_from_beam_fieldtype(\n            beam_fieldtype.array_type.element_type))\n  elif type_info == \"map_type\":\n    output_arrow_type = _make_arrow_map(beam_fieldtype.map_type)\n  elif type_info == \"row_type\":\n    schema = beam_fieldtype.row_type.schema\n    # Note schema id and options are handled at the arrow field level, they are\n    # added at field-level metadata.\n    output_arrow_type = pa.struct(\n        [_arrow_field_from_beam_field(field) for field in schema.fields])\n  elif type_info == \"logical_type\":\n    # TODO(https://github.com/apache/beam/issues/23817): Add support for logical\n    # types.\n    raise NotImplementedError(\n        \"Beam logical types are not currently supported \"\n        \"in arrow_type_compatibility.\")\n  else:\n    raise ValueError(f\"Unrecognized type_info: {type_info!r}\")\n\n  return output_arrow_type\n\n\nclass PyarrowBatchConverter(BatchConverter):\n  def __init__(self, element_type: RowTypeConstraint):\n    super().__init__(pa.Table, element_type)\n    self._beam_schema = typing_to_runner_api(element_type).row_type.schema\n    arrow_schema = arrow_schema_from_beam_schema(self._beam_schema)\n\n    self._arrow_schema = arrow_schema\n\n  @staticmethod\n  def from_typehints(element_type,","sourceCodeStart":271,"sourceCodeEnd":307,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/typehints/arrow_type_compatibility.py#L271-L307","documentation":"Beam logical types (schema_pb2 logical_type) are not implemented in the arrow conversion layer (tracked as beam issue 23817). When _arrow_type_from_beam_fieldtype encounters type_info == 'logical_type' it raises NotImplementedError.","triggerScenarios":"Converting a Beam schema containing a logical-type field (e.g. types registered via beam.typehints with logical type constructors, or SqlType wrappers) to arrow via PyarrowBatchConverter or _arrow_field_from_beam_fieldtype.","commonSituations":"Using Beam SQL logical types (DATE, TIME, DECIMAL via zetasql wrappers) in a PCollection then batching to pa.Table; cross-SDK schemas carrying logical types; third-party logical type registrations.","solutions":["Replace the logical type field with its underlying representation (e.g. use datetime/int64/string base types)","Convert logical values to primitive types before the arrow batch conversion","Track/upgrade for Beam support of issue 23817 (logical type support in arrow conversion)"],"exampleFix":"// before\nbeam.Row(ts=LogicalType('sql:DATE', value))\n// after\nbeam.Row(ts=datetime.date(value))  # plain date, arrow-supported","handlingStrategy":"fallback","validationCode":"if ft.WhichOneof('type_info') == 'logical_type':\n    raise TypeError('logical types not supported for arrow conversion; flatten to primitives first')","typeGuard":"def is_logical_type(ft) -> bool:\n    return ft.WhichOneof('type_info') == 'logical_type'","tryCatchPattern":"try:\n    arrow_type = _arrow_type_from_beam_fieldtype(ft)\nexcept NotImplementedError:\n    ft = to_underlying_primitive(ft)\n    arrow_type = _arrow_type_from_beam_fieldtype(ft)","preventionTips":["Avoid Beam logical-type fields in PCollections destined for arrow batching","Represent logical values as base types (datetime, int, str)","Track beam issue 23817 for when support lands"],"tags":["python","apache-beam","pyarrow","unsupported-feature"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}