apache/flink · error · NullPointerException

Hadoop input split must not be null

Error message

Hadoop input split must not be null

What it means

Thrown by the mapreduce-API HadoopInputSplit constructor when the wrapped InputSplit argument is null. This is a defensive precondition: a null split cannot be serialized via the Writable protocol the wrapper depends on, so construction refuses it immediately rather than NPE'ing later during write/read.

Source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/wrapper/HadoopInputSplit.java:53

 * InputSplit}.
 */
@Internal
public class HadoopInputSplit extends LocatableInputSplit {

    private static final long serialVersionUID = 6119153593707857235L;

    private final Class<? extends org.apache.hadoop.mapreduce.InputSplit> splitType;

    private transient org.apache.hadoop.mapreduce.InputSplit mapreduceInputSplit;

    public HadoopInputSplit(
            int splitNumber,
            org.apache.hadoop.mapreduce.InputSplit mapreduceInputSplit,
            JobContext jobContext) {
        super(splitNumber, (String) null);

        if (mapreduceInputSplit == null) {
            throw new NullPointerException("Hadoop input split must not be null");
        }
        if (!(mapreduceInputSplit instanceof Writable)) {
            throw new IllegalArgumentException("InputSplit must implement Writable interface.");
        }
        this.splitType = mapreduceInputSplit.getClass();
        this.mapreduceInputSplit = mapreduceInputSplit;
    }

    // ------------------------------------------------------------------------
    //  Properties
    // ------------------------------------------------------------------------

    public org.apache.hadoop.mapreduce.InputSplit getHadoopInputSplit() {
        return mapreduceInputSplit;
    }

    @Override
    public String[] getHostnames() {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the Hadoop InputFormat.getSplits() never returns null entries; filter or reject nulls at the source.
2. Guard construction sites that build HadoopInputSplit so a null split is logged and skipped rather than wrapped.
3. If the null comes from a third-party InputFormat, patch or wrap it to never emit null splits.
4. In tests, always pass a real (or mocked, non-null) InputSplit to the constructor.

Example fix

// before
for (InputSplit s : splits) {
    hadoopInputSplits[i] = new HadoopInputSplit(i, s, jobContext);
}
// after — skip null splits defensively
for (InputSplit s : splits) {
    if (s == null) continue;
    hadoopInputSplits[i++] = new HadoopInputSplit(i, s, jobContext);
}

// Before constructing the wrapper, assert the split is non-null
if (mapreduceInputSplit == null) {
    throw new IllegalArgumentException("Refusing to wrap a null Hadoop InputSplit");
}
new HadoopInputSplit(splitNumber, mapreduceInputSplit, jobContext);

java.util.Objects.requireNonNull(mapreduceInputSplit, "Hadoop InputSplit must not be null")

When it happens

Trigger: Produced when new HadoopInputSplit(splitNumber, mapreduceInputSplit, jobContext) is called with mapreduceInputSplit == null — e.g. Hadoop's InputFormat.getSplits() returned a list containing a null element, or user code constructed the wrapper with a null split.

Common situations: A buggy/custom InputFormat whose getSplits() emits null entries; a wrapper built manually in tests with a null split; upstream split list corruption where a split entry failed to initialize.

Related errors

• InputSplit must implement Writable interface.
• Unable to instantiate the Hadoop InputSplit
• Mapper may not be null.
• JobConf may not be null.
• The collection contains a null element

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/6f0935cb26a86de2. Report an issue: GitHub.