apache/flink · error · NullKeyFieldException

{}

Error message

{}

What it means

compareToReference in PojoComparator throws NullKeyFieldException (message is just the field name) when comparing against a stored reference raises NullPointerException - i.e. the reference record's key field (or the candidate's) is null during a reference-based comparison used by sort and join algorithms.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/PojoComparator.java:259

        }
        return true;
    }

    @Override
    public int compareToReference(TypeComparator<T> referencedComparator) {
        PojoComparator<T> other = (PojoComparator<T>) referencedComparator;

        int i = 0;
        try {
            for (; i < this.keyFields.length; i++) {
                int cmp = this.comparators[i].compareToReference(other.comparators[i]);
                if (cmp != 0) {
                    return cmp;
                }
            }
            return 0;
        } catch (NullPointerException npex) {
            throw new NullKeyFieldException(this.keyFields[i].toString());
        }
    }

    @Override
    public int compare(T first, T second) {
        int i = 0;
        for (; i < keyFields.length; i++) {
            int cmp =
                    comparators[i].compare(
                            accessField(keyFields[i], first), accessField(keyFields[i], second));
            if (cmp != 0) {
                return cmp;
            }
        }

        return 0;
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Normalize null key fields to defaults or sentinel values upstream of sort/join operators
2. Filter null-keyed records before the operation that triggers comparison
3. Use the field name in the exception to add the missing null-handling for that specific field

Example fix

// before
d.sortPartition(pojo -> pojo.getScore(), Order.ASCENDING); // score may be null

// after
d.map(p -> p.withScore(p.getScore() == null ? 0L : p.getScore()))
 .sortPartition(pojo -> pojo.getScore(), Order.ASCENDING);

// before sort/join on a POJO field
boolean hasNullKeys = collection.stream().anyMatch(p -> p.getKeyField() == null);
if (hasNullKeys) {
    throw new IllegalArgumentException("Null key fields present; normalize before sorting");
}

static boolean isSortSafe(MyPojo p) {
    return p != null && p.getKeyField() != null;
}

try {
    comparator.compareToReference(ref);
} catch (NullKeyFieldException e) {
    // e.getMessage() is the field name whose value is null
    throw new IllegalArgumentException("Null key field: " + e.getMessage(), e);
}

When it happens

Trigger: Range-partitioning/sort/join paths that call setReference + compareToReference on POJO records where keyFields[i] evaluates to null; the NPE escapes the loop and is converted to NullKeyFieldException carrying the field name only.

Common situations: Records with null key fields flowing into sortPartition, co-group, or interval joins; null-producingUDF outputs entering keyed/sorted operators; same root cause as hash() null keys but hit on the reference-comparison path.

Related errors

• A NullPointerException occurred while accessing a key field
• Comparator {} specifies an invalid length for the normalized
• Cannot copy serializer
• Unable to access field {} on object {}
• Field {fieldNumber} is null, but expected to hold a key.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/76568f4fe4ed9b9a. Report an issue: GitHub.