{"record":{"id":"75a24ac10ed41377","repo":"apache/beam","slug":"unable-to-convert-avro-type-to-a-beam-schema","errorCode":null,"errorMessage":"Unable to convert {avro_type} to a Beam schema.","messagePattern":"Unable to convert (.+?) to a Beam schema\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/avroio.py","lineNumber":621,"sourceCode":"    return schema_pb2.FieldType(\n        array_type=schema_pb2.ArrayType(\n            element_type=avro_type_to_beam_type(avro_type['items'])))\n  elif type_name == 'map':\n    return schema_pb2.FieldType(\n        map_type=schema_pb2.MapType(\n            key_type=schema_pb2.FieldType(atomic_type=schema_pb2.STRING),\n            value_type=avro_type_to_beam_type(avro_type['values'])))\n  elif type_name == 'record':\n    return schema_pb2.FieldType(\n        row_type=schema_pb2.RowType(\n            schema=schema_pb2.Schema(\n                fields=[\n                    schemas.schema_field(\n                        f['name'], avro_type_to_beam_type(f['type']))\n                    for f in avro_type['fields']\n                ])))\n  else:\n    raise ValueError(f'Unable to convert {avro_type} to a Beam schema.')\n\n\ndef avro_schema_to_beam_schema(\n    avro_schema: _AvroSchemaType) -> schema_pb2.Schema:\n  beam_type = avro_type_to_beam_type(avro_schema)\n  if isinstance(avro_schema, dict) and avro_schema['type'] == 'record':\n    return beam_type.row_type.schema\n  else:\n    return schema_pb2.Schema(fields=[schemas.schema_field('record', beam_type)])\n\n\ndef avro_dict_to_beam_row(\n    avro_schema: _AvroSchemaType,\n    beam_schema: schema_pb2.Schema) -> Callable[[Any], Any]:\n  if isinstance(avro_schema, str):\n    return avro_dict_to_beam_row({'type': avro_schema})\n  if avro_schema['type'] == 'record':\n    to_row = avro_value_to_beam_value(","sourceCodeStart":603,"sourceCodeEnd":639,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/avroio.py#L603-L639","documentation":"avroio.avro_type_to_beam_type maps Avro schema type descriptors (dicts like {'type':'record',...}, 'string', ['null','int'], etc.) to Beam schema FieldTypes. Avro's type system includes constructs Beam schemas cannot express directly; when the descriptor is none of the supported forms (record, enum, array, map, union, named primitive, or string primitive name), the function raises ValueError with the offending avro_type.","triggerScenarios":"Calling avro_schema_to_beam_schema / avro_type_to_beam_type on an Avro schema containing an unsupported type descriptor — e.g. a dict whose 'type' is 'fixed', a malformed dict without a recognized 'type' key, or a union/list form that is not a simple nullable union.","commonSituations":"Reading exotic Avro files (fixed-size binary fields, complex unions of 3+ branches) and piping them through Beam schema conversion; hand-written or third-party-generated .avsc schemas with constructs Beam's mapper does not handle.","solutions":["Pre-process the Avro schema: replace unsupported types (e.g. 'fixed' with 'bytes', 3+ branch unions with single nullable unions) before conversion","Read the file as raw records (ParseAllFromAvro yields dicts) and Map them into beam.Row objects yourself instead of relying on automatic schema conversion","Check the Beam version — newer releases support more Avro constructs; upgrade apache-beam","Validate the schema dict's 'type' field is one Beam supports (record/enum/array/map/union/string primitives)"],"exampleFix":"// before\nschema = {'type': 'record', 'name': 'R', 'fields': [{'name': 'h', 'type': {'type': 'fixed', 'name': 'F', 'size': 16}}]}\navroio.avro_schema_to_beam_schema(schema)  # ValueError\n// after\nschema = {'type': 'record', 'name': 'R', 'fields': [{'name': 'h', 'type': 'bytes'}]}\nbeam_schema = avroio.avro_schema_to_beam_schema(schema)","handlingStrategy":"try-catch","validationCode":"SUPPORTED = {'record','enum','array','map','union','string','bytes','int','long','float','double','boolean','null'}\nassert all(isinstance(t, str) and t in SUPPORTED or isinstance(t, (dict, list)) for t in flat_types(avro_schema))","typeGuard":null,"tryCatchPattern":"try:\n    beam_schema = avroio.avro_schema_to_beam_schema(avro_schema)\nexcept ValueError as e:\n    logging.error('Avro type not supported: %s', e)\n    # fall back to reading raw dicts instead of schema'd rows","preventionTips":["Pre-process .avsc schemas to remove 'fixed' and complex (3+ branch) unions","Verify conversion works in a unit test before running the full pipeline","Upgrade apache-beam for wider Avro coverage"],"tags":["python","apache-beam","avro","schema","conversion"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}