{"record":{"id":"7447dea8bf28fc93","repo":"apache/beam","slug":"you-are-using-avro-io-with-fastavro-default-with-beam-on","errorCode":null,"errorMessage":"You are using Avro IO with fastavro (default with Beam on Python 3), but supplying a schema parsed by avro-python3. Please change the schema to a dict.","messagePattern":"You are using Avro IO with fastavro \\(default with Beam on Python 3\\), but supplying a schema parsed by avro-python3\\. Please change the schema to a dict\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/io/avroio.py","lineNumber":467,"sourceCode":"          self._sink.shard_name_template)\n\n    return records | beam.io.iobase.Write(self._sink)\n\n  def display_data(self):\n    return {'sink_dd': self._sink}\n\n\ndef _create_avro_sink(\n    file_path_prefix,\n    schema,\n    codec,\n    file_name_suffix,\n    num_shards,\n    shard_name_template,\n    mime_type,\n    triggering_frequency=60):\n  if \"class 'avro.schema\" in str(type(schema)):\n    raise ValueError(\n        'You are using Avro IO with fastavro (default with Beam on '\n        'Python 3), but supplying a schema parsed by avro-python3. '\n        'Please change the schema to a dict.')\n  return _FastAvroSink(\n      file_path_prefix,\n      schema,\n      codec,\n      file_name_suffix,\n      num_shards,\n      shard_name_template,\n      mime_type,\n      triggering_frequency)\n\n\nclass _BaseAvroSink(filebasedsink.FileBasedSink):\n  \"\"\"A base for a sink for avro files. \"\"\"\n  def __init__(\n      self,","sourceCodeStart":449,"sourceCodeEnd":485,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/io/avroio.py#L449-L485","documentation":"apache_beam.io.avroio._create_avro_sink chooses the sink implementation based on the Avro library available. When fastavro is installed (the default for Beam on Python 3), schemas must be dicts (fastavro-style); supplying a schema object parsed by the avro-python3 library (its type string starts with \"class 'avro.schema\") raises ValueError because fastavro cannot consume avro-python3 schema objects.","triggerScenarios":"Creating WriteToAvro/_create_avro_sink with schema=<avro.schema.Schema object from avro-python3's schema.parse()> while Beam is using the fastavro-based sink.","commonSituations":"Mixing avro-python3 and fastavro in one project; loading a .avsc file with the wrong library; older Beam code that used avro-python3 being run under a Python 3 Beam install that defaults to fastavro; dependencies pulling in avro-python3 transitively and code paths branching on it.","solutions":["Load the schema as a plain dict: json.load(open('schema.avsc')) instead of avro.schema.parse()","Uninstall/avoid avro-python3 and rely on fastavro (Beam's Python 3 default)","Construct the schema dict inline with {'type': 'record', 'name': ..., 'fields': [...]}","If you must use avro-python3 schemas, force the non-fastavro sink by ensuring fastavro is not installed (not recommended)"],"exampleFix":"// before\nimport avro.schema\nschema = avro.schema.parse(open('schema.avsc').read())\nWriteToAvro('out.avro', schema=schema)  # ValueError\n// after\nimport json\nschema = json.load(open('schema.avsc'))\nWriteToAvro('out.avro', schema=schema)","handlingStrategy":"type-guard","validationCode":"is_bad = \"class 'avro.schema\" in str(type(schema))\nassert not is_bad, 'use a dict schema for fastavro'","typeGuard":"def is_fastavro_schema(schema):\n    return isinstance(schema, dict) and 'type' in schema\n\ndef is_avro_python3_schema(schema):\n    return \"class 'avro.schema\" in str(type(schema))","tryCatchPattern":"try:\n    sink = WriteToAvro(path, schema=schema)\nexcept ValueError as e:\n    import json\n    if isinstance(schema, str):\n        schema = json.loads(schema)\n    else:\n        schema = schema.to_json()\n    sink = WriteToAvro(path, schema=schema)","preventionTips":["Load .avsc files with json.load, not avro-python3's schema.parse()","Standardize on fastavro in your project's dependencies","Add a CI check that no code path produces avro-python3 Schema objects for AvroIO"],"tags":["python","apache-beam","avro","fastavro","type-mismatch"],"backgroundTag":"incompatible-source-type","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}