apache/flink · critical · RuntimeException

Unable to instantiate the Hadoop InputSplit

Error message

Unable to instantiate the Hadoop InputSplit

What it means

Thrown during Java deserialization of the mapreduce-API HadoopInputSplit. readObject() narrows the stored splitType to a Writable subclass and rebuilds the instance via WritableFactories.newInstance(writableSplit). If the class cannot be instantiated (no no-arg constructor, not loadable, not Writable-factory-enabled) the RuntimeException aborts split recovery and the task fails.

Source

Thrown at flink-connectors/flink-hadoop-compatibility/src/main/java/org/apache/flink/api/java/hadoop/mapreduce/wrapper/HadoopInputSplit.java:101

    private void writeObject(ObjectOutputStream out) throws IOException {
        // serialize the parent fields and the final fields
        out.defaultWriteObject();

        // write the input split
        ((Writable) mapreduceInputSplit).write(out);
    }

    private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException {
        // read the parent fields and the final fields
        in.defaultReadObject();

        try {
            Class<? extends Writable> writableSplit = splitType.asSubclass(Writable.class);
            mapreduceInputSplit =
                    (org.apache.hadoop.mapreduce.InputSplit)
                            WritableFactories.newInstance(writableSplit);
        } catch (Exception e) {
            throw new RuntimeException("Unable to instantiate the Hadoop InputSplit", e);
        }

        ((Writable) mapreduceInputSplit).readFields(in);
    }
}

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Ensure the concrete Writable InputSplit class is on the TaskManager classpath via the user-code jar.
2. Verify the split class has a public no-arg constructor used by WritableFactories.newInstance.
3. If needed, register a WritableFactory for the split class via WritableFactories.registerFactory(splitClass, factory).
4. Keep the split class name consistent under any shading so asSubclass/forName resolve at runtime.

Example fix

// before — custom writable split without default constructor
public class MySplit extends InputSplit implements Writable {
    public MySplit(Path p) { ... }
}
// after — add no-arg constructor for WritableFactories
public class MySplit extends InputSplit implements Writable {
    public MySplit() {}
    public MySplit(Path p) { ... }
}

// Before shipping, verify the split class is a Writable instantiable via WritableFactories
Class<?> splitType = myMapreduceInputSplit.getClass();
if (!org.apache.hadoop.io.Writable.class.isAssignableFrom(splitType))
    throw new IllegalStateException(splitType.getName() + " is not Writable");
try {
    Object probe = org.apache.hadoop.io.WritableFactories.newInstance(
        splitType.asSubclass(org.apache.hadoop.io.Writable.class));
    if (probe == null) throw new IllegalStateException("WritableFactories returned null");
} catch (Exception e) {
    throw new IllegalStateException("Cannot instantiate split " + splitType.getName(), e);
}

org.apache.hadoop.io.Writable.class.isAssignableFrom(myMapreduceInputSplit.getClass())

When it happens

Trigger: Triggered on the TaskManager when a serialized mapreduce HadoopInputSplit is rebuilt and splitType.asSubclass(Writable.class) or WritableFactories.newInstance(writableSplit) throws — e.g. the split class is not on the TaskManager classpath, has no accessible no-arg constructor, or was not registered with a WritableFactory.

Common situations: A custom mapreduce InputSplit (Writable) whose class is missing from the shipped jar; split class with no default constructor; class shading/relocation mismatch between client serialization and TaskManager; Hadoop version where the split class changed its factory registration.

Related errors

• Unable to instantiate Hadoop InputSplit
• Unable to instantiate the hadoop input format
• Unable to find key class.
• Unable to find value class.
• Unable to instantiate the hadoop output format

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/c0ff13f505836b8c. Report an issue: GitHub.