What causes "%s is not nullable in Map field %s" in apache/beam?

Declare map values nullable: Schema.FieldType.map(keyType, valueType.withNullable(true))

apache/beam · error · java.lang.IllegalArgumentException

%s is not nullable in Map field %s

Error message

%s is not nullable in Map field %s

What it means

For MAP fields, Beam enforces nullability of map values (keys must be non-null/non-nullable by schema rules). When processMap captures a map entry whose value is null and the value FieldType is declared non-nullable, it throws IllegalArgumentException.

Source

Thrown at sdks/java/core/src/main/java/org/apache/beam/sdk/values/RowUtils.java:520

        RowPosition rowPosition,
        FieldType keyType,
        FieldType valueType,
        Map<Object, Object> valueMap,
        RowFieldMatcher matcher) {
      Map<Object, Object> retValue = null;
      FieldOverride override = override(rowPosition);
      if (override != null) {
        valueMap = (Map<Object, Object>) override.getOverrideValue();
      }

      if (valueMap != null) {
        RowPosition elementPosition = rowPosition.withMapQualifier();

        retValue = Maps.newHashMapWithExpectedSize(valueMap.size());
        for (Entry<Object, Object> kv : valueMap.entrySet()) {
          if (kv.getValue() == null) {
            if (!valueType.getNullable()) {
              throw new IllegalArgumentException(
                  String.format(
                      "%s is not nullable in Map field %s", valueType, rowPosition.descriptor));
            }
            retValue.put(matcher.match(this, keyType, elementPosition, kv.getKey()), null);
          } else {
            retValue.put(
                matcher.match(this, keyType, elementPosition, kv.getKey()),
                matcher.match(this, valueType, elementPosition, kv.getValue()));
          }
        }
      }
      return retValue;
    }

    @Override
    public Object processLogicalType(
        RowPosition rowPosition, LogicalType logicalType, Object value, RowFieldMatcher matcher) {
      Object retValue = null;

View on GitHub (pinned to 12126d8942)

Solutions

Declare map values nullable: Schema.FieldType.map(keyType, valueType.withNullable(true))
Remove or default null values before Row creation (Maps.filterValues(m, Objects::nonNull) or fill defaults)
Fix upstream producers so map values are never null

Example fix

// before
Map<String, Integer> counts = ...; // may contain null values
// after
counts.values().removeIf(Objects::isNull);

Defensive patterns

Strategy: validation

Validate before calling

map.values().removeIf(Objects::isNull); // or use valueType.withNullable(true)

Prevention

Never put null values in maps destined for Rows with non-nullable value types
Use Maps.filterValues to sanitize input
Declare map value nullability explicitly to match data

When it happens

Trigger: RowUtils.processMap iterating a Map field during schema attach/match; any entry with kv.getValue() == null while valueType.getNullable() is false.

Common situations: Maps assembled from sparse data where lookups returned null; schema evolved to non-nullable values; deserializing older records containing null map values.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/343e577fe7b67185. Report an issue: GitHub.