apache/hadoop · error · ClassCastException

Child value classes fail to agree

Error message

Child value classes fail to agree

What it means

MultiFilterRecordReader, the base of override-style joins (OverrideRecordReader), emits a single child value per record instead of a TupleWritable, so all children must share one value class. createValue() throws ClassCastException("Child value classes fail to agree") when the child RecordReaders return different value classes. It fires on the first createValue() call during task setup.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/join/MultiFilterRecordReader.java:101

      jc.reset(iterkey);
      if (jc.flush(ivalue)) {
        WritableUtils.cloneInto(key, jc.key());
        WritableUtils.cloneInto(value, emit(ivalue));
        return true;
      }
      jc.clear();
    }
    return false;
  }

  /** {@inheritDoc} */
  @SuppressWarnings("unchecked") // Explicit check for value class agreement
  public V createValue() {
    if (null == valueclass) {
      final Class<?> cls = kids[0].createValue().getClass();
      for (RecordReader<K,? extends V> rr : kids) {
        if (!cls.equals(rr.createValue().getClass())) {
          throw new ClassCastException("Child value classes fail to agree");
        }
      }
      valueclass = cls.asSubclass(Writable.class);
      ivalue = createInternalValue();
    }
    return (V) ReflectionUtils.newInstance(valueclass, null);
  }

  /**
   * Return an iterator returning a single value from the tuple.
   * @see MultiFilterDelegationIterator
   */
  protected ResetableIterator<V> getDelegate() {
    return new MultiFilterDelegationIterator();
  }

  /**
   * Proxy the JoinCollector, but include callback to emit.

View on GitHub (pinned to 2add963021)

Solutions

1. Re-emit the mismatched source with a map-only job so all children share one value class
2. Use the same InputFormat and serialization for every source in the override expression
3. Switch the outer node from override to inner/outer when a TupleWritable value is acceptable, since tuple joins tolerate per-child value classes

Example fix

// before: override requires one shared value class, sources differ
job.set("mapred.join.expr", "override(tbl(SequenceFileInputFormat,/old),tbl(SequenceFileInputFormat,/new))");

// after: normalize values, or use outer(...) which emits TupleWritable
job.set("mapred.join.expr",
    CompositeInputFormat.compose("outer", SequenceFileInputFormat.class, "/old", "/new"));

FileSystem fs = FileSystem.get(conf);
Set<Class<?>> valueClasses = new HashSet<Class<?>>();
for (Path p : joinSources) {
  SequenceFile.Reader r = new SequenceFile.Reader(fs, p, conf);
  try {
    valueClasses.add(r.getValueClass());
  } finally {
    r.close();
  }
}
if (valueClasses.size() > 1) {
  throw new IOException("override join requires one shared value class: " + valueClasses);
}

static boolean valuesAgree(ComposableRecordReader<?> a, ComposableRecordReader<?> b) {
  return a.createValue().getClass().equals(b.createValue().getClass());
}

try {
  V val = rr.createValue();
} catch (ClassCastException e) {
  throw new IOException("override children must share one value class; "
      + "normalize the sources or use inner/outer", e);
}

When it happens

Trigger: An override(...) join whose tbl(...) sources produce different value classes, e.g. SequenceFile Text values against a custom Writable; any custom reader extending MultiFilterRecordReader whose children disagree on their value class.

Common situations: Using override to co-group sources written by different jobs or serializations; one source regenerated with a new schema while the others kept the old one.

Related errors

• Child key classes fail to agree
• Invalid split type:{}
• Invalid split type:{}
• Type mismatch in key from map: expected {keyClassName}, rece
• Type mismatch in value from map: expected {valueClassName},

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/7792c70dafe402be. Report an issue: GitHub.