apache/flink · error · RuntimeException

Comparator {className} specifies an invalid length for the n

Error message

Comparator {className} specifies an invalid length for the normalized key: {len}

What it means

TupleComparatorBase's constructor precomputes normalized-key metadata by asking each field comparator for getNormalizeKeyLen(). A length < 0 is invalid (negative means 'infinite' only via supportsNormalizedKeyOnReference/ other flags, and a negative len here breaks key-space math), so it throws RuntimeException naming the comparator class and the bad length. This is a contract violation by a custom field-level TypeComparator.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/TupleComparatorBase.java:93

        for (int i = 0; i < this.keyPositions.length; i++) {
            TypeComparator<?> k = this.comparators[i];

            // as long as the leading keys support normalized keys, we can build up the composite
            // key
            if (k.supportsNormalizedKey()) {
                if (i == 0) {
                    // the first comparator decides whether we need to invert the key direction
                    inverted = k.invertNormalizedKey();
                } else if (k.invertNormalizedKey() != inverted) {
                    // if a successor does not agree on the inversion direction, it cannot be part
                    // of the normalized key
                    break;
                }

                nKeys++;
                final int len = k.getNormalizeKeyLen();
                if (len < 0) {
                    throw new RuntimeException(
                            "Comparator "
                                    + k.getClass().getName()
                                    + " specifies an invalid length for the normalized key: "
                                    + len);
                }
                this.normalizedKeyLengths[i] = len;
                nKeyLen += len;

                if (nKeyLen < 0) {
                    // overflow, which means we are out of budget for normalized key space anyways
                    nKeyLen = Integer.MAX_VALUE;
                    break;
                }
            } else {
                break;
            }
        }
        this.numLeadingNormalizableKeys = nKeys;

View on GitHub (pinned to 2f3c205e92)

Solutions

  1. Fix the custom comparator: getNormalizeKeyLen() must return >= 0 whenever it advertises a normalizable key; if it cannot produce a normalized key, override supportsNormalizedKey() to return false instead of returning a negative length.
  2. Add a unit test asserting getNormalizeKeyLen() >= 0 for the custom comparator under all configurations.
  3. As a workaround while fixing, use a built-in comparator for that field, or wrap the custom one so normalized-key support is disabled.

Example fix

// before
public class MyComparator extends TypeComparator<Long> {
    @Override public int getNormalizeKeyLen() { return -1; }
    @Override public boolean supportsNormalizedKey() { return true; }
}

// after
public class MyComparator extends TypeComparator<Long> {
    @Override public boolean supportsNormalizedKey() { return false; } // no normalized key
    // or: @Override public int getNormalizeKeyLen() { return 8; } // Long.SIZE / Byte.SIZE
}
Defensive patterns

Strategy: validation

Validate before calling

public static void checkFieldComparators(TypeComparator<?>[] fieldComparators) {
    for (TypeComparator<?> k : fieldComparators) {
        if (k.supportsNormalizedKey() && k.getNormalizeKeyLen() < 0) {
            throw new IllegalStateException(k.getClass().getName()
                + " advertises a normalized key but returns length " + k.getNormalizeKeyLen());
        }
    }
}

Try / catch

try {
    new TupleComparator<>(keyPositions, fieldComparators, fieldSerializers);
} catch (RuntimeException e) {
    // message names the offending comparator class and length; fix getNormalizeKeyLen()
}

Prevention

When it happens

Trigger: Supplying a custom TypeComparator as a field comparator whose getNormalizeKeyLen() returns a negative value while it still participates in the normalized key computation; TupleComparator/TupleComparatorBase constructor then throws during comparator creation (client- or setup-time).

Common situations: User implements a custom comparator and returns -1 from getNormalizeKeyLen() unconditionally (copying a template) while supportsNormalizedKey() returns true. Flink version change altered expectations around normalize-key lengths. Third-party comparator implementations that never were exercised with tuple composition.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/29fabd08e3c7c4c0. Report an issue: GitHub.