{"record":{"id":"c6f04fedc12bf681","repo":"apache/flink","slug":"field-fieldnumber-is-null-but-expected-to-hold-c6f04f","errorCode":null,"errorMessage":"Field {fieldNumber} is null, but expected to hold a key.","messagePattern":"Field (.+?) is null, but expected to hold a key\\.","errorType":"exception","errorClass":"NullKeyFieldException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/TupleComparatorBase.java","lineNumber":181,"sourceCode":"    //  Comparator Methods\n    // --------------------------------------------------------------------------------------------\n\n    @Override\n    public int compareToReference(TypeComparator<T> referencedComparator) {\n        TupleComparatorBase<T> other = (TupleComparatorBase<T>) referencedComparator;\n\n        int i = 0;\n        try {\n            for (; i < this.keyPositions.length; i++) {\n                @SuppressWarnings(\"unchecked\")\n                int cmp = this.comparators[i].compareToReference(other.comparators[i]);\n                if (cmp != 0) {\n                    return cmp;\n                }\n            }\n            return 0;\n        } catch (NullPointerException npex) {\n            throw new NullKeyFieldException(keyPositions[i]);\n        } catch (IndexOutOfBoundsException iobex) {\n            throw new KeyFieldOutOfBoundsException(keyPositions[i]);\n        }\n    }\n\n    @SuppressWarnings(\"unchecked\")\n    @Override\n    public int compareSerialized(DataInputView firstSource, DataInputView secondSource)\n            throws IOException {\n        if (deserializedFields1 == null) {\n            instantiateDeserializationUtils();\n        }\n\n        int i = 0;\n        try {\n            for (; i < serializers.length; i++) {\n                deserializedFields1[i] =\n                        serializers[i].deserialize(deserializedFields1[i], firstSource);","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/TupleComparatorBase.java#L163-L199","documentation":"TupleComparatorBase.compareToReference(TypeComparator other) compares the stored reference records of two comparators field-by-field via each field comparator's compareToReference. If a reference field value is null, the field comparator throws NullPointerException, which is converted to NullKeyFieldException(keyPositions[i]). The record previously stored as reference has a null in a key position, so reference-based comparison cannot proceed.","triggerScenarios":"A sort/hash-match operator calls setReference on records, later compareToReference runs; one of the reference records had null at a key position -> NPE inside the field comparator -> NullKeyFieldException. Happens with null values in sort/join/group key fields regardless of the record path (reference vs candidate).","commonSituations":"Null keys from dirty source data, outer-join outputs, optional fields becoming keys. Same data-quality class as error 691 but hit through the reference-comparison path.","solutions":["Eliminate nulls from key fields before records enter the keyed/sorted operator (filter or substitute defaults).","Choose key fields that are guaranteed non-null by the schema; validate with a unit test containing edge-case nulls.","For fields that legitimately allow null, sort on a null-aware wrapper value instead of raw key positions."],"exampleFix":"// before\nenv.fromElements(Tuple2.of((Long) null, \"a\"), Tuple2.of(1L, \"b\"))\n   .groupBy(0).reduce(...); // NullKeyFieldException(0)\n\n// after\nenv.fromElements(Tuple2.of((Long) null, \"a\"), Tuple2.of(1L, \"b\"))\n   .filter(t -> t.f0 != null)\n   .groupBy(0).reduce(...);","handlingStrategy":"validation","validationCode":"public static <T extends Tuple> boolean referenceSafe(T record, int[] keyPositions) {\n    for (int p : keyPositions) {\n        if (record.getField(p) == null) return false;\n    }\n    return true;\n}\n// only setReference/compareToReference after referenceSafe(record, keyPositions)","typeGuard":null,"tryCatchPattern":"try {\n    int cmp = a.compareToReference(b);\n} catch (NullKeyFieldException e) {\n    // reference record has a null key field: route to null-key handling or reject record\n}","preventionTips":["Guarantee non-null key fields before records enter reference-based matching.","Audit join/enrichment steps that can emit null keys.","Include null-key fixtures in operator tests."],"tags":["tuple","null-key","comparator","reference-compare","data-quality"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}