{"record":{"id":"1fd97c1e22fdd068","repo":"apache/flink","slug":"field-fieldnumber-is-null-but-expected-to-hold","errorCode":null,"errorMessage":"Field {fieldNumber} is null, but expected to hold a key.","messagePattern":"Field (.+?) is null, but expected to hold a key\\.","errorType":"exception","errorClass":"NullKeyFieldException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/TupleComparator.java","lineNumber":140,"sourceCode":"    }\n\n    @SuppressWarnings(\"unchecked\")\n    @Override\n    public void putNormalizedKey(T value, MemorySegment target, int offset, int numBytes) {\n        int i = 0;\n        try {\n            for (; i < this.numLeadingNormalizableKeys && numBytes > 0; i++) {\n                int len = this.normalizedKeyLengths[i];\n                len = numBytes >= len ? len : numBytes;\n                this.comparators[i].putNormalizedKey(\n                        value.getFieldNotNull(this.keyPositions[i]), target, offset, len);\n                numBytes -= len;\n                offset += len;\n            }\n        } catch (NullFieldException nfex) {\n            throw new NullKeyFieldException(nfex);\n        } catch (NullPointerException npex) {\n            throw new NullKeyFieldException(this.keyPositions[i]);\n        }\n    }\n\n    @Override\n    public int extractKeys(Object record, Object[] target, int index) {\n        int localIndex = index;\n        for (int i = 0; i < comparators.length; i++) {\n            localIndex +=\n                    comparators[i].extractKeys(\n                            ((Tuple) record).getField(keyPositions[i]), target, localIndex);\n        }\n        return localIndex - index;\n    }\n\n    public TypeComparator<T> duplicate() {\n        return new TupleComparator<T>(this);\n    }\n}","sourceCodeStart":122,"sourceCodeEnd":158,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/TupleComparator.java#L122-L158","documentation":"TupleComparator.putNormalizedKey writes the leading key fields' normalized forms into a memory segment for fast sorting. If a key field's value is null, the underlying comparator throws NullPointerException, which this method converts into NullKeyFieldException(keyPositions[i]) with message 'Field {fieldNumber} is null, but expected to hold a key.' Sorting/partitioning requires non-null keys; nulls in key fields are rejected.","triggerScenarios":"Sorting or range-partitioning tuple records where the field at a key position is null, and putNormalizedKey is invoked (normalized-key path of the sort algorithm) -> NPE inside the field comparator -> NullKeyFieldException. Nulls in keys of Tuple1<String> with null element are the classic case.","commonSituations":"Upstream data contains nulls in the field chosen as sort/group key (dirty source data, left-join misses, JSON with missing attributes). Schema evolution adds nullability to a key column. Unit tests using null placeholders in key fields.","solutions":["Filter or transform null keys before the sort/keyed operation: records with null keys cannot participate in ordering.","Replace nulls with a sentinel that sorts as intended (e.g. empty string, Long.MIN_VALUE) at the projection step, and document the sentinel.","If nulls are legitimate, sort on a wrapped type with a null-aware comparator instead of raw tuple key positions."],"exampleFix":"// before\nDataStream<Tuple2<Long,String>> s = env.fromElements(\n    Tuple2.of(1L, (String) null), Tuple2.of(2L, \"b\"));\ns.sortPartition(1, Order.ASCENDING); // NullKeyFieldException(1)\n\n// after\nDataStream<Tuple2<Long,String>> s = env.fromElements(\n    Tuple2.of(1L, (String) null), Tuple2.of(2L, \"b\"))\n    .filter(t -> t.f1 != null); // or map null -> \"\"\ns.sortPartition(1, Order.ASCENDING);","handlingStrategy":"validation","validationCode":"public static boolean hasNonNullKeys(Tuple t, int[] keyPositions) {\n    for (int p : keyPositions) {\n        if (t.getField(p) == null) return false;\n    }\n    return true;\n}\n// stream.filter(t -> hasNonNullKeys(t, keyPositions)) before sorting/partitioning","typeGuard":"public static boolean hasSortSafeKeys(Tuple2<Long,String> t) {\n    return t.f1 != null; // key field non-null\n}","tryCatchPattern":"try {\n    comparator.putNormalizedKey(value, segment, off, len);\n} catch (org.apache.flink.api.common.typeutils.NullKeyFieldException e) {\n    // field index in message: null key encountered; divert record to a null-handling path\n}","preventionTips":["Filter or default null key fields before sort/partition operators.","Avoid choosing nullable columns as sort/group keys.","Add null-bearing records to data-quality tests for keyed pipelines."],"tags":["tuple","null-key","normalized-key","sorting","data-quality"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}