apache/flink · error · NullPointerException

Hadoop JobConf must not be null when input split is configur

Error message

Hadoop JobConf must not be null when input split is configurable.

What it means

Thrown by the HadoopInputSplit constructor when the InputSplit implements Hadoop's Configurable interface (meaning it needs a JobConf to configure itself after deserialization) but the provided JobConf is null. Configurable splits cannot function without configuration, so the constructor rejects the null JobConf.

Source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapred/wrapper/HadoopInputSplit.java:62

    private final Class<? extends org.apache.hadoop.mapred.InputSplit> splitType;

    private transient org.apache.hadoop.mapred.InputSplit hadoopInputSplit;

    @Nullable private transient JobConf jobConf;

    public HadoopInputSplit(
            int splitNumber,
            org.apache.hadoop.mapred.InputSplit hInputSplit,
            @Nullable JobConf jobconf) {
        super(splitNumber, (String) null);

        if (hInputSplit == null) {
            throw new NullPointerException("Hadoop input split must not be null");
        }

        if (needsJobConf(hInputSplit) && jobconf == null) {
            throw new NullPointerException(
                    "Hadoop JobConf must not be null when input split is configurable.");
        }

        this.splitType = hInputSplit.getClass();

        this.jobConf = jobconf;
        this.hadoopInputSplit = hInputSplit;
    }

    // ------------------------------------------------------------------------
    //  Properties
    // ------------------------------------------------------------------------

    @Override
    public String[] getHostnames() {
        try {
            return this.hadoopInputSplit.getLocations();
        } catch (IOException e) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Always pass a non-null JobConf when constructing HadoopInputSplit for Configurable InputSplits.
2. In deserialization paths (readFields), ensure the JobConf is deserialized and set before the split is used.
3. Use HadoopInputSplit.updateJobConf(jobConf) after deserialization if the JobConf was not available at construction time.

Example fix

// before — Configurable split with null JobConf
HadoopInputSplit split = new HadoopInputSplit(0, configurableInputSplit, null);
// after — provide the JobConf
HadoopInputSplit split = new HadoopInputSplit(0, configurableInputSplit, jobConf);
// or update after deserialization
split.updateJobConf(jobConf);

// Check if the split is Configurable and ensure JobConf is provided
if (hInputSplit instanceof org.apache.hadoop.conf.Configurable && jobConf == null) {
    throw new IllegalArgumentException(
        "Configurable InputSplit requires a non-null JobConf");
}
HadoopInputSplit split = new HadoopInputSplit(splitNumber, hInputSplit, jobConf);

public static boolean needsJobConf(org.apache.hadoop.mapred.InputSplit split) {
    return split instanceof org.apache.hadoop.conf.Configurable;
}

When it happens

Trigger: Constructing a HadoopInputSplit(splitNumber, hInputSplit, null) where hInputSplit implements org.apache.hadoop.conf.Configurable — typically after deserialization when the JobConf was not passed to the split wrapper.

Common situations: Deserializing a HadoopInputSplit without restoring its JobConf; custom split distribution code that strips the JobConf; test code that creates Configurable splits without providing configuration.

Related errors

• Hadoop input split must not be null
• Reducer may not be null.
• JobConf may not be null.
• The broadcast input name may not be null.
• The broadcast input root operator may not be null.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/86680707abfb3456. Report an issue: GitHub.