{"record":{"id":"7d6be07b0e24f0f0","repo":"apache/beam","slug":"unsupported-atomic-type-0","errorCode":null,"errorMessage":"Unsupported atomic type: {0}","messagePattern":"Unsupported atomic type: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/typehints/arrow_type_compatibility.py","lineNumber":272,"sourceCode":"        _arrow_field_from_beam_fieldtype(beam_map_type.value_type))\n\n  def _arrow_map_to_beam_map(arrow_map_type):\n    return schema_pb2.MapType(\n        key_type=_beam_fieldtype_from_arrow_field(arrow_map_type.key_field),\n        value_type=_beam_fieldtype_from_arrow_field(arrow_map_type.item_field))\n\n\ndef _arrow_type_from_beam_fieldtype(\n    beam_fieldtype: schema_pb2.FieldType,\n) -> Tuple[pa.DataType, Optional[Dict[bytes, bytes]]]:\n  # Note this function is not concerned with beam_fieldtype.nullable, as\n  # nullability is a property of the Field in Arrow.\n  type_info = beam_fieldtype.WhichOneof(\"type_info\")\n  if type_info == 'atomic_type':\n    try:\n      output_arrow_type = ATOMIC_TYPE_TO_PYARROW[beam_fieldtype.atomic_type]\n    except KeyError:\n      raise ValueError(\n          \"Unsupported atomic type: {0}\".format(beam_fieldtype.atomic_type))\n  elif type_info == \"array_type\":\n    output_arrow_type = pa.list_(\n        _arrow_field_from_beam_fieldtype(\n            beam_fieldtype.array_type.element_type))\n  elif type_info == \"map_type\":\n    output_arrow_type = _make_arrow_map(beam_fieldtype.map_type)\n  elif type_info == \"row_type\":\n    schema = beam_fieldtype.row_type.schema\n    # Note schema id and options are handled at the arrow field level, they are\n    # added at field-level metadata.\n    output_arrow_type = pa.struct(\n        [_arrow_field_from_beam_field(field) for field in schema.fields])\n  elif type_info == \"logical_type\":\n    # TODO(https://github.com/apache/beam/issues/23817): Add support for logical\n    # types.\n    raise NotImplementedError(\n        \"Beam logical types are not currently supported \"","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/typehints/arrow_type_compatibility.py#L254-L290","documentation":"_arrow_type_from_beam_fieldtype maps Beam atomic types to pyarrow types via the ATOMIC_TYPE_TO_PYARROW table. If the Beam FieldType's atomic_type has no pyarrow equivalent in the table, it raises ValueError 'Unsupported atomic type'.","triggerScenarios":"Converting a Beam schema containing an atomic type absent from the mapping (e.g. Beam logical/bytes-backed specializations or proto-defined atomic types not in ATOMIC_TYPE_TO_PYARROW) through _arrow_field_from_beam_fieldtype, _make_arrow_map, or PyarrowBatchConverter construction.","commonSituations":"Using Beam logical types (e.g. its sql/LogicalType wrappers) with arrow batch conversion; schemas exchanged from other Beam SDKs (Java/Go) with exotic atomic types; adding new Beam atomic types while on an old Beam/pyarrow version.","solutions":["Use a Beam atomic type with a direct pyarrow equivalent (string, int64, double, etc.)","Upgrade apache-beam so the mapping table includes your type","Pre-convert the field to a supported type in your pipeline (e.g. cast to int64/string) before schema conversion"],"exampleFix":"// before\nbeam.Row(x=beam_logical_type_value)  # atomic type not in ATOMIC_TYPE_TO_PYARROW\n// after\nbeam.Row(x=int(beam_logical_type_value))  # plain int64, supported","handlingStrategy":"type-guard","validationCode":"import apache_beam.typehints.schema as schema_pb2\nfrom apache_beam.typehints.arrow_type_compatibility import ATOMIC_TYPE_TO_PYARROW\nif ft.WhichOneof('type_info') == 'atomic_type' and ft.atomic_type not in ATOMIC_TYPE_TO_PYARROW:\n    raise TypeError(f'atomic type {ft.atomic_type} has no pyarrow mapping; use a primitive type')","typeGuard":"def arrow_supported(ft) -> bool:\n    from apache_beam.typehints.arrow_type_compatibility import ATOMIC_TYPE_TO_PYARROW\n    return ft.WhichOneof('type_info') != 'atomic_type' or ft.atomic_type in ATOMIC_TYPE_TO_PYARROW","tryCatchPattern":"try:\n    arrow_type = _arrow_type_from_beam_fieldtype(ft)\nexcept ValueError as e:\n    if 'Unsupported atomic type' in str(e):\n        ft = cast_to_supported_primitive(ft)\n        arrow_type = _arrow_type_from_beam_fieldtype(ft)\n    else:\n        raise","preventionTips":["Restrict row types to primitive fields when planning pyarrow batching","Convert custom/logical atomic types to base primitives upstream","Keep apache-beam updated so new atomic types are mapped"],"tags":["python","apache-beam","pyarrow","type-conversion"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}