apache/flink · error · IllegalArgumentException

InputSplit must implement Writable interface.

Error message

InputSplit must implement Writable interface.

What it means

Thrown by the mapreduce-API HadoopInputSplit constructor when the wrapped InputSplit does not implement Hadoop's Writable interface. The wrapper serializes the split across the network via the Writable protocol (write/readFields), so a non-Writable split cannot be shipped to TaskManagers and construction rejects it.

Source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/wrapper/HadoopInputSplit.java:56

public class HadoopInputSplit extends LocatableInputSplit {

    private static final long serialVersionUID = 6119153593707857235L;

    private final Class<? extends org.apache.hadoop.mapreduce.InputSplit> splitType;

    private transient org.apache.hadoop.mapreduce.InputSplit mapreduceInputSplit;

    public HadoopInputSplit(
            int splitNumber,
            org.apache.hadoop.mapreduce.InputSplit mapreduceInputSplit,
            JobContext jobContext) {
        super(splitNumber, (String) null);

        if (mapreduceInputSplit == null) {
            throw new NullPointerException("Hadoop input split must not be null");
        }
        if (!(mapreduceInputSplit instanceof Writable)) {
            throw new IllegalArgumentException("InputSplit must implement Writable interface.");
        }
        this.splitType = mapreduceInputSplit.getClass();
        this.mapreduceInputSplit = mapreduceInputSplit;
    }

    // ------------------------------------------------------------------------
    //  Properties
    // ------------------------------------------------------------------------

    public org.apache.hadoop.mapreduce.InputSplit getHadoopInputSplit() {
        return mapreduceInputSplit;
    }

    @Override
    public String[] getHostnames() {
        try {
            return mapreduceInputSplit.getLocations();
        } catch (Exception e) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Make the custom InputSplit class implement org.apache.hadoop.io.Writable (provide write and readFields).
2. Use a built-in Hadoop split type (e.g. FileSplit) which already implements Writable.
3. If the InputFormat only produces non-Writable splits, wrap/adapt the split or choose a different InputFormat compatible with the Writable requirement.
4. Verify the split instance passed to the constructor is the concrete Writable type, not a raw interface.

Example fix

// before — split not Writable
public class MySplit extends org.apache.hadoop.mapreduce.InputSplit {
    // missing write/readFields
}
// after — implement Writable
public class MySplit extends org.apache.hadoop.mapreduce.InputSplit
        implements org.apache.hadoop.io.Writable {
    @Override public void write(DataOutput out) throws IOException { ... }
    @Override public void readFields(DataInput in) throws IOException { ... }
}

// Before constructing the wrapper, assert the split is Writable
if (!(mapreduceInputSplit instanceof org.apache.hadoop.io.Writable)) {
    throw new IllegalArgumentException("InputSplit " + mapreduceInputSplit.getClass().getName()
        + " must implement org.apache.hadoop.io.Writable to be shipped by Flink");
}
new HadoopInputSplit(splitNumber, mapreduceInputSplit, jobContext);

mapreduceInputSplit instanceof org.apache.hadoop.io.Writable

When it happens

Trigger: Produced when new HadoopInputSplit(splitNumber, mapreduceInputSplit, jobContext) is given a mapreduceInputSplit that is not instanceof Writable — e.g. a custom InputSplit that omits org.apache.hadoop.io.Writable, or a split from an InputFormat that uses a non-Writable split type (incompatible with this wrapper).

Common situations: A custom mapreduce InputSplit implemented without extending/implementing Writable; using a newer Hadoop API split that is not Writable; misunderstanding that Flink's mapreduce wrapper only supports Writable splits (unlike the mapred wrapper, which uses WritableFactories on mapred.InputSplit).

Related errors

• Hadoop input split must not be null
• Unable to instantiate the Hadoop InputSplit
• The given class is no subclass of {Writable.class.getName()}
• Unable to instantiate Hadoop InputSplit
• Could not get KeyValue pair.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/745e15af68138347. Report an issue: GitHub.