apache/flink · error · RuntimeException

JSON format doesn't support to serialize map data with null

Error message

JSON format doesn't support to serialize map data with null keys. You can drop null key entries or encode null in literals by specifying %s option.

What it means

Thrown by RowDataToJsonConverters.createMapConverter when a map being serialized to JSON has a null key and mapNullKeyMode is FAIL (the default). JSON object keys cannot be null, so the format offers three modes via the 'json.map-null-key.mode' option: FAIL (default), DROP (skip entry), LITERAL (write the string from 'json.map-null-key.literal').

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/RowDataToJsonConverters.java:306

                node.removeAll();
            }

            MapData map = (MapData) object;
            ArrayData keyArray = map.keyArray();
            ArrayData valueArray = map.valueArray();
            int numElements = map.size();
            for (int i = 0; i < numElements; i++) {
                String fieldName = null;
                if (keyArray.isNullAt(i)) {
                    // when map key is null
                    switch (mapNullKeyMode) {
                        case LITERAL:
                            fieldName = mapNullKeyLiteral;
                            break;
                        case DROP:
                            continue;
                        case FAIL:
                            throw new RuntimeException(
                                    String.format(
                                            "JSON format doesn't support to serialize map data with null keys. "
                                                    + "You can drop null key entries or encode null in literals by specifying %s option.",
                                            JsonFormatOptions.MAP_NULL_KEY_MODE.key()));
                        default:
                            throw new RuntimeException(
                                    "Unsupported map null key mode. Validator should have checked that.");
                    }
                } else {
                    fieldName = keyArray.getString(i).toString();
                }

                Object value = valueGetter.getElementOrNull(valueArray, i);
                node.set(fieldName, valueConverter.convert(mapper, node.get(fieldName), value));
            }

            return node;
        };

View on GitHub (pinned to 2f3c205e92)

Solutions

Set 'json.map-null-key.mode' = 'DROP' to skip null-key entries
Set 'json.map-null-key.mode' = 'LITERAL' together with 'json.map-null-key.literal' = 'null' (or another marker) to encode null keys as a string
Filter null keys out in the query (WHERE k IS NOT NULL) or COALESCE the key column before it reaches the map

Example fix

// before
WITH ('connector'='kafka', 'format'='json')
-- map {null: 'x'} throws

// after
WITH ('connector'='kafka', 'format'='json',
  'json.map-null-key.mode'='LITERAL', 'json.map-null-key.literal'='null')

Defensive patterns

Strategy: validation

Validate before calling

// Decide policy up front in the WITH clause instead of the FAIL default:
// 'json.map-null-key.mode' = 'DROP' | 'LITERAL' + 'json.map-null-key.literal'

Try / catch

catch (RuntimeException e) on null-map-key message — switch map-null-key mode option; the failing record itself is usually legitimate data.

Prevention

Always set 'json.map-null-key.mode' explicitly for sink tables fed by joins/aggregations
COALESCE nullable keys before they become map keys
Test sinks with null-key sample rows

When it happens

Trigger: Writing a row whose MAP contains a null key with default options. Commonly from aggregations that produce null keys (e.g. GROUP BY on a nullable column, LEFT JOIN results) feeding a JSON sink such as Kafka with 'format'='json'.

Common situations: Null keys from outer joins or optional grouping attributes; upstream data quality issues producing null map keys; switching a sink from a format that tolerated null keys to JSON.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/06164596b0f70a05. Report an issue: GitHub.