apache/hadoop · error · IOException

Failed tuple init

Error message

Failed tuple init

What it means

TupleWritable serializes the class name of every element (TupleWritable.write, TupleWritable.java:174). On read, TupleWritable.readFields loads each class via Class.forName and rethrows failure as IOException('Failed tuple init', e) — this instance wraps ClassNotFoundException: an element class named in the stream could not be loaded on the reading JVM.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/join/TupleWritable.java:207

    values = new Writable[card];
    readBitSet(in, card, written);
    Class<? extends Writable>[] cls = new Class[card];
    try {
      for (int i = 0; i < card; ++i) {
        cls[i] = Class.forName(Text.readString(in)).asSubclass(Writable.class);
      }
      for (int i = 0; i < card; ++i) {
        if (cls[i].equals(NullWritable.class)) {
          values[i] = NullWritable.get();
        } else {
          values[i] = cls[i].newInstance();
        }
        if (has(i)) {
          values[i].readFields(in);
        }
      }
    } catch (ClassNotFoundException e) {
      throw new IOException("Failed tuple init", e);
    } catch (IllegalAccessException e) {
      throw new IOException("Failed tuple init", e);
    } catch (InstantiationException e) {
      throw new IOException("Failed tuple init", e);
    }
  }

  /**
   * Record that the tuple contains an element at the position provided.
   */
  void setWritten(int i) {
    written.set(i);
  }

  /**
   * Record that the tuple does not contain an element at the position
   * provided.
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Ship the jar containing the element Writable class with the job: use -libjars on hadoop jar, job.setJarByClass(MyWritable.class), or place the jar in the job's lib directory / DistributedCache
2. Ensure the exact class name (package included) present at write time exists at read time — avoid renaming or relocating Writable classes across versions
3. For shaded deployments, keep serialized class names stable (exclude Writable packages from relocation) or use a stable wrapper Writable
4. Catch IOException around reader.next() and inspect getCause(): ClassNotFoundException names the missing class — then verify with 'hadoop classpath' / task logs which jar is absent

Example fix

# before
hadoop jar myjoin.jar com.acme.JoinDriver /in /out   # custom Writable in acme-common.jar not shipped

# after
hadoop jar myjoin.jar com.acme.JoinDriver -libjars acme-common.jar /in /out

// also in driver:
job.setJarByClass(com.acme.common.MyTupleElement.class); // ensure task classpath contains it

static void requireTupleClassesOnClasspath(Path sampleData, Configuration conf) throws IOException {
  // read one record locally and fail fast if any element class is missing
  try (java.io.DataInputStream in = new java.io.DataInputStream(
      sampleData.getFileSystem(conf).open(sampleData))) {
    TupleWritable t = new TupleWritable(new Writable[0]);
    t.readFields(in); // throws 'Failed tuple init' with ClassNotFoundException as cause if missing
  } catch (IOException e) {
    if (e.getCause() instanceof ClassNotFoundException)
      throw new IllegalStateException("Missing class on classpath: " + e.getCause().getMessage()
        + " — add its jar via -libjars / job.setJarByClass", e);
    throw e;
  }
}

try { tuple.readFields(in); } catch (IOException e) { if (e.getCause() instanceof ClassNotFoundException) { /* log missing class name, fix classpath, fail task with actionable message */ } throw e; }

When it happens

Trigger: A mapper/reducer reading join output (TupleWritable) where an element class (custom Writable) is missing from the task classpath: jar not shipped with the job (job.setJarByClass omitted / libjars missing), class renamed between write and read, shaded/relocated class names in the serialized stream, or classes written by a newer job version read by an older one.

Common situations: Custom Writable key/value classes in join data flows; 'hadoop jar' without -libjars; Oozie/Spark-generated data read by a MapReduce join with a different libset; class relocation by maven-shade-plugin changing binary names; client-side jar present but task-side classpath lacking the class.

Related errors

• Type mismatch in key from map: expected {keyClassName}, rece
• Type mismatch in value from map: expected {valueClassName},
• Uninitialized InputSplit
• Too many splits
• Child key classes fail to agree

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e94e4f097f4c0c49. Report an issue: GitHub.