What causes "Avro only supports maps with string keys" in apache/beam?

Change the map key type to STRING in the Beam schema before conversion (e.g. encode integer keys as strings).

apache/beam · error · IllegalArgumentException

Avro only supports maps with string keys

Error message

Avro only supports maps with string keys

What it means

Avro's MAP type only allows string keys, while Beam Schema allows maps with arbitrary key types. During FieldType -> Avro conversion, if the map's key type is not a string type, the converter throws IllegalArgumentException because no valid Avro representation exists.

Source

Thrown at sdks/java/extensions/avro/src/main/java/org/apache/beam/sdk/extensions/avro/schemas/utils/AvroUtils.java:1255

        }
        break;

      case ARRAY:
      case ITERABLE:
        baseType =
            org.apache.avro.Schema.createArray(
                getFieldSchema(
                    checkNotNull(fieldType.getCollectionElementType()), fieldName, namespace));
        break;

      case MAP:
        if (checkNotNull(fieldType.getMapKeyType()).getTypeName().isStringType()) {
          // Avro only supports string keys in maps.
          baseType =
              org.apache.avro.Schema.createMap(
                  getFieldSchema(checkNotNull(fieldType.getMapValueType()), fieldName, namespace));
        } else {
          throw new IllegalArgumentException("Avro only supports maps with string keys");
        }
        break;

      case ROW:
        baseType = toAvroSchema(checkNotNull(fieldType.getRowSchema()), fieldName, namespace);
        break;

      default:
        throw new IllegalArgumentException("Unexpected type " + fieldType);
    }
    return fieldType.getNullable() ? ReflectData.makeNullable(baseType) : baseType;
  }

  private static final Map<org.apache.avro.Schema, Function<Number, ? extends Number>>
      NUMERIC_CONVERTERS =
          ImmutableMap.of(
              org.apache.avro.Schema.create(org.apache.avro.Schema.Type.INT), Number::intValue,
              org.apache.avro.Schema.create(org.apache.avro.Schema.Type.LONG), Number::longValue,

View on GitHub (pinned to 12126d8942)

Solutions

Change the map key type to STRING in the Beam schema before conversion (e.g. encode integer keys as strings).
Represent the map as an ARRAY of ROW<'key','value'> records instead of a MAP, which Avro supports with any key type.
Reject/normalize non-string-key maps upstream in the pipeline before writing to Avro.
Catch IllegalArgumentException and use a schema-transformation fallback (array-of-pairs) for non-string-key maps.

Example fix

// before
Field.of("counts", FieldType.map(FieldType.INT64, FieldType.STRING))
// after
Field.of("counts", FieldType.map(FieldType.STRING, FieldType.STRING))
// or an array of key/value rows:
Field.of("counts", FieldType.array(FieldType.row(Field.of("k", FieldType.STRING), Field.of("v", FieldType.STRING))))

Defensive patterns

Strategy: validation

Validate before calling

static boolean isAvroCompatibleMap(FieldType ft) {
  return ft.getTypeName() != TypeName.MAP
      || checkNotNull(ft.getMapKeyType()).getTypeName().isStringType();
}

Try / catch

try {
  org.apache.avro.Schema s = AvroUtils.toAvroSchema(beamSchema);
} catch (IllegalArgumentException e) {
  if (e.getMessage().contains("maps with string keys")) {
    org.apache.avro.Schema s2 = AvroUtils.toAvroSchema(mapToArrayOfPairs(beamSchema));
  } else { throw e; }
}

Prevention

Constrain Beam schemas to STRING-keyed maps whenever Avro output is possible.
Prefer array-of-ROW key/value encodings for non-string keys.
Validate map key types when building schemas from external sources.
Document the Avro string-key restriction for schema authors.

When it happens

Trigger: Calling AvroUtils.getFieldSchema/toAvroSchema on a FieldType of TypeName.MAP whose getMapKeyType() is not STRING (e.g. Map<Integer, X> or MAP<BYTES,...> in the Beam schema).

Common situations: Converting Beam rows from sources like BigQuery/Flink that allow non-string map keys into Avro for Parquet writes; portable pipeline expansion where a coder maps non-string keys to Avro.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/69ca4784cdf162e6. Report an issue: GitHub.