apache/beam · error · TypeError

Only strings allowed as map keys when converting to AVRO…

Error message

Only strings allowed as map keys when converting to AVRO, found {beam_type}

What it means

apache_beam raises this TypeError in beam_type_to_avro_type when a Beam Schema contains a map type whose key type is not the STRING atomic type. AVRO map keys must be strings, so any Beam map with non-string keys (e.g. int or bytes keys) cannot be represented in an AVRO schema. The check fires in the map_type branch while converting a Beam schema to an AVRO schema.

Solutions

  1. Change the Beam schema so every map field uses string keys (coerce keys to str before writing).
  2. Convert non-string-keyed maps to an ARRAY of ROW(key, value) fields, which AVRO can represent.
  3. Write to a format that supports non-string map keys (e.g. Parquet) instead of AVRO.
  4. Catch TypeError around the conversion and raise a clearer schema-design error at pipeline-construction time.

Example fix

// before
schema = beam.Row(m=beam.MapType[int, str])  # int keys
WriteToAvro('out.avro', schema=schema)
// after
row = beam.Row(m={str(k): v for k, v in my_int_keyed_map.items()})  # string keys
WriteToAvro('out.avro', schema=beam.schema_from(row))
Defensive patterns

Strategy: validation

Validate before calling

from apache_beam import schema_pb2
def validate_string_map_keys(schema: schema_pb2.Schema):
    for f in schema.fields:
        if f.type.type_info == schema_pb2.FieldType.MAP_TYPE and \
           f.type.map_type.key_type.atomic_type != schema_pb2.STRING:
            raise ValueError(f"Map field {f.name!r} must have string keys for AVRO")

Try / catch

try:
    avro_schema = beam_schema_to_avro_schema(beam_schema)
except TypeError as e:
    if 'map keys' in str(e):
        # coerce map keys or redesign field, then retry
        ...
    raise

Prevention

When it happens

Trigger: Calling apache_beam.io.avroio.beam_schema_to_avro_schema() (directly or via AvroIO with schema generation) on a schema whose schema_pb2.Schema has a MAP field whose key_type.atomic_type is not STRING (e.g. map<int, str>).

Common situations: Pipelines that build Beam rows programmatically with non-string map keys and write them to AVRO sinks; schemas inferred from Python dicts with int keys; converting BigQuery/Parquet-derived schemas that allow non-string map keys to AVRO.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/da2f46b492c66b20. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/io/avroio.py:732

  if type_info == "atomic_type":
    avro_primitive = BEAM_PRIMITIVES_TO_AVRO_PRIMITIVES[beam_type.atomic_type]
    avro_type = [
        avro_primitive, 'null'
    ] if beam_type.nullable else avro_primitive
    return {'type': avro_type}
  elif type_info == "array_type":
    return {
        'type': 'array',
        'items': unnest_primitive_type(beam_type.array_type.element_type)
    }
  elif type_info == "iterable_type":
    return {
        'type': 'array',
        'items': unnest_primitive_type(beam_type.iterable_type.element_type)
    }
  elif type_info == "map_type":
    if beam_type.map_type.key_type.atomic_type != schema_pb2.STRING:
      raise TypeError(
          f'Only strings allowed as map keys when converting to AVRO, '
          f'found {beam_type}')
    return {
        'type': 'map',
        'values': unnest_primitive_type(beam_type.map_type.element_type)
    }
  elif type_info == "row_type":
    return {
        'type': 'record',
        'name': beam_type.row_type.schema.id,
        'fields': [{
            'name': field.name, 'type': unnest_primitive_type(field.type)
        } for field in beam_type.row_type.schema.fields],
    }
  else:
    raise ValueError(f"Unconvertable type: {beam_type}")

View on GitHub (pinned to 12126d8942)