apache/flink · error · RuntimeException

Fail to serialize at field: %s.

Error message

Fail to serialize at field: %s.

What it means

Thrown by the ROW converter in RowDataToJsonConverters when any per-field serializer throws while writing one field of a row to JSON. It is a wrapper: the message names the field and the cause holds the real failure (unsupported type, bad value for the target JSON node, variant failure, etc.). Diagnose the cause chain for the named field.

Source

Thrown at flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/RowDataToJsonConverters.java:362

            ObjectNode node;
            // reuse could be a NullNode if last record is null.
            if (reuse == null || reuse.isNull()) {
                node = mapper.createObjectNode();
            } else {
                node = (ObjectNode) reuse;
            }
            RowData row = (RowData) value;
            for (int i = 0; i < fieldCount; i++) {
                String fieldName = fieldNames[i];
                try {
                    Object field = fieldGetters[i].getFieldOrNull(row);
                    if (field != null || !ignoreNullFields) {
                        node.set(
                                fieldName,
                                fieldConverters[i].convert(mapper, node.get(fieldName), field));
                    }
                } catch (Throwable t) {
                    throw new RuntimeException(
                            String.format("Fail to serialize at field: %s.", fieldName), t);
                }
            }
            return node;
        };
    }

    private RowDataToJsonConverter wrapIntoNullableConverter(RowDataToJsonConverter converter) {
        return (mapper, reuse, object) -> {
            if (object == null) {
                return mapper.getNodeFactory().nullNode();
            }

            return converter.convert(mapper, reuse, object);
        };
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Inspect the cause of the RuntimeException to find the real converter failure for the named field
2. Fix the specific nested cause: adjust types, set 'json.map-null-key.mode', remove RAW columns, or sanitize values before the sink
3. Test the sink schema with a representative sample of rows (including edge cases: nulls, empty maps, variants) before production
4. Add a dead-letter/quarantine path for records that legitimately cannot be serialized

Example fix

// before: nested map with null key hits JSON sink
INSERT INTO kafka_sink SELECT * FROM agg_view;

// after: sanitize before sinking
INSERT INTO kafka_sink SELECT id, MAP_VALUES(...) /* or COALESCE keys */ FROM agg_view;

catch (RuntimeException e) — extract field name from message, unwrap cause, apply the nested fix (map-null-key mode, type change, variant quarantine); do not retry blindly.

When it happens

Trigger: Any field-level serialization error: a VARIANT value failing convertVariant (1406), a map with null keys inside a nested row (1408), an unsupported type nested in a ROW, or a value whose conversion throws. Raised at runtime on the offending record during JSON sink writes.

Common situations: Schema drift between computing pipeline and sink schema; nested maps with null keys from joins/aggregations; RAW or exotic types nested inside ROW columns written to JSON.

Related errors

• Fail to deserialize at field: %s.
• Field at index %s must be of type byte[], but was %s
• Row arity of record ({}) does not match this serializer's fi
• Unable to clone instance of %s.
• Illegal JSON object data...

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/b1a1bb0fd9c1d30e. Report an issue: GitHub.