apache/hadoop · error · IOException

Failed split init

Error message

Failed split init

What it means

Error "Failed split init" thrown in apache/hadoop.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/CompositeInputSplit.java:163

    int card = WritableUtils.readVInt(in);
    if (splits == null || splits.length != card) {
      splits = new InputSplit[card];
    }
    Class<? extends InputSplit>[] cls = new Class[card];
    try {
      for (int i = 0; i < card; ++i) {
        cls[i] =
          Class.forName(Text.readString(in)).asSubclass(InputSplit.class);
      }
      for (int i = 0; i < card; ++i) {
        splits[i] = ReflectionUtils.newInstance(cls[i], null);
        SerializationFactory factory = new SerializationFactory(conf);
        Deserializer deserializer = factory.getDeserializer(cls[i]);
        deserializer.open((DataInputStream)in);
        splits[i] = (InputSplit)deserializer.deserialize(splits[i]);
      }
    } catch (ClassNotFoundException e) {
      throw new IOException("Failed split init", e);
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Check the wrapped IOException cause: usually a child InputSplit failed to deserialize; verify the split class is on the classpath and its readFields() matches write().
2. Confirm all child InputSplit implementations are Writable and registered consistently between job submission and task execution.

Example fix

// add the jar containing the custom InputSplit to the job
job.addFileToClassPath(new Path("/libs/mysplits.jar"));

When it happens

Trigger: Thrown by CompositeInputSplit.readFields() when deserializing a child split fails because the child split class cannot be instantiated or its readFields throws. Verify the child InputSplit classes are on the classpath and have a public no-arg constructor.

Common situations: Custom InputSplit class missing from the task classpath; incompatible serializer versions between client and cluster.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/573edcd42307afe4. Report an issue: GitHub.