apache/flink · error · UnsupportedOperationException

Record serialization with leading normalized keys not suppor

Error message

Record serialization with leading normalized keys not supported.

What it means

NullAwareComparator.writeWithKeyNormalization() always throws UnsupportedOperationException. Normalized-key serialization would need to write the null-flag byte at a position the wrapped comparator's key-normalization protocol cannot express (the wrapper prepends its own null byte in putNormalizedKey), so this TypeComparator feature is intentionally unimplemented.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/NullAwareComparator.java:193

            if (record == null) {
                target.putBoolean(offset, false);
                // write padding
                for (int j = 0; j < numBytes - 1; j++) {
                    target.put(offset + 1 + j, (byte) 0);
                }
            }
            // write a non-null byte with key
            else {
                target.putBoolean(offset, true);
                // write key
                wrappedComparator.putNormalizedKey(record, target, offset + 1, numBytes - 1);
            }
        }
    }

    @Override
    public void writeWithKeyNormalization(T record, DataOutputView target) throws IOException {
        throw new UnsupportedOperationException(
                "Record serialization with leading normalized keys not supported.");
    }

    @Override
    public T readWithKeyDenormalization(T reuse, DataInputView source) throws IOException {
        throw new UnsupportedOperationException(
                "Record deserialization with leading normalized keys not supported.");
    }

    @Override
    public boolean invertNormalizedKey() {
        return wrappedComparator.invertNormalizedKey();
    }

    @Override
    public TypeComparator<T> duplicate() {
        return new NullAwareComparator<T>(wrappedComparator.duplicate(), order);
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Check comparator.supportsSerializationWithKeyNormalization() before using key-normalization serialization paths
2. Route sorting through the composite comparator (RowComparator), which implements these methods at composite level
3. Make the wrapped field non-null in the schema (disable nullability) so the plain comparator is used

Example fix

// before
comparator.writeWithKeyNormalization(record, target);

// after
if (comparator.supportsSerializationWithKeyNormalization()) {
    comparator.writeWithKeyNormalization(record, target);
} else {
    serializer.serialize(record, target);
}

if (!comparator.supportsSerializationWithKeyNormalization()) {
    serializer.serialize(record, target); // plain path
} else {
    comparator.writeWithKeyNormalization(record, target);
}

When it happens

Trigger: A sort/min-max algorithm or runtime path that requires 'serialization with key normalization' (normalizing keys that are not fixed-length normalized keys) calls this method on a null-aware field comparator; the feature check supportsSerializationWithKeyNormalization() returns false, so callers should have probed it first.

Common situations: Custom operators implementing large-scale sort with key-normalized serialization; runtime paths configured with comparators whose supportsNormalizedKey() is false falling back to key-normalized serialization on a null-aware field.

Related errors

• Record deserialization with leading normalized keys not supp
• Comparator does not support null-aware serialized comparison
• Record serialization with leading normalized keys not suppor
• Record deserialization with leading normalized keys not supp
• Workaround hack.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/e13526f00bfe21d3. Report an issue: GitHub.