apache/flink · error · NullFieldException

Field {fieldIdx} is null, but expected to hold a value.

Error message

Field {fieldIdx} is null, but expected to hold a value.

What it means

TupleSerializer.serialize writes each tuple field with its field serializer. When a field's value is null and the field serializer throws NullPointerException from serialize, it is converted into NullFieldException(i) with message 'Field {fieldIdx} is null, but expected to hold a value.' Standard Flink tuple serializers do not support null fields (only tuple fields of type-nullable wrappers handled by specific serializers); a null field value therefore aborts serialization of the whole record.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/TupleSerializer.java:141

            return null;
        }

        for (int i = 0; i < arity; i++) {
            Object copy = fieldSerializers[i].copy((Object) from.getField(i), reuse.getField(i));
            reuse.setField(copy, i);
        }

        return reuse;
    }

    @Override
    public void serialize(T value, DataOutputView target) throws IOException {
        for (int i = 0; i < arity; i++) {
            Object o = value.getField(i);
            try {
                fieldSerializers[i].serialize(o, target);
            } catch (NullPointerException npex) {
                throw new NullFieldException(i, npex);
            }
        }
    }

    @Override
    public T deserialize(DataInputView source) throws IOException {
        T tuple = instantiateRaw();
        for (int i = 0; i < arity; i++) {
            Object field = fieldSerializers[i].deserialize(source);
            tuple.setField(field, i);
        }
        return tuple;
    }

    @Override
    public T deserialize(T reuse, DataInputView source) throws IOException {
        for (int i = 0; i < arity; i++) {
            Object field = fieldSerializers[i].deserialize(reuse.getField(i), source);

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Replace nulls with type-appropriate defaults (empty StringValue, 0L) before serialization, or filter incomplete records.
2. If nulls must be represented, use plain Java wrapper types (String, Long) with Flink's standard TypeInfo so nullability is encoded, or use a custom TypeInformation whose serializers handle nulls (e.g. SqlTimestamp/nullable wrappers).
3. Identify the offending field from the NullFieldException field index and fix that specific upstream producer.

Example fix

// before
Tuple2<StringValue, LongValue> t = Tuple2.of(null, new LongValue(1));
// serializer.serialize(t, out) -> NullFieldException(0)

// after
Tuple2<StringValue, LongValue> t = Tuple2.of(new StringValue(""), new LongValue(1));
serializer.serialize(t, out);

public static boolean allFieldsNonNull(Tuple t) {
    for (int i = 0; i < t.getArity(); i++) {
        if (t.getField(i) == null) return false;
    }
    return true;
}
// stream.filter(RecordUtil::allFieldsNonNull) or default-fill before serialize

public static boolean tupleSerializableSafe(Tuple2<String,Long> t) {
    return t.f0 != null && t.f1 != null;
}

try {
    serializer.serialize(value, out);
} catch (org.apache.flink.api.java.typeutils.runtime.NullFieldException e) {
    // e.getFieldIndex() names the null field; fix that producer or default-fill it
}

When it happens

Trigger: Serializing a tuple (network shuffle, checkpointing, writing to a sink) where value.getField(i) is null for a primitive-typed field serializer (e.g. StringValue, LongValue-based serializers) whose serialize(null) NPEs -> NullFieldException(i) naming the field index.

Common situations: Source data with missing/optional attributes mapped into tuple fields using Value-type serializers (StringValue etc.) instead of null-tolerant ones (String field via StringSerializer handles null differently; Value types never do). Left-join or enrichment steps producing nulls. Reusing a TupleX of Value objects where some elements were left null by constructor defaults.

Related errors

• Tuple0 cannot take any data, as it has zero fields.
• Field {fieldNumber} is null, but expected to hold a key.
• Field {fieldNumber} is null, but expected to hold a key.
• Cannot instantiate tuple.
• Cannot deserialize and unwrap accumulators properly.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/460923ba183aa41b. Report an issue: GitHub.