apache/hadoop · error · IOException

Type mismatch in value from map: expected {valueClassName},

Error message

Type mismatch in value from map: expected {valueClassName}, received {actualValueClassName}

What it means

The sorting collector type-checks every value passed to OutputCollector.collect() / Context.write() with an exact getClass() comparison against the configured map-output value class (mapreduce.map.output.value.class, set via Job.setMapOutputValueClass). When map() emits a value whose runtime class - including a subclass - differs from the configured one, the map task fails with this IOException on the first offending record.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java:1102

            sortSpillException);
      }
    }

    /**
     * Serialize the key, value to intermediate storage.
     * When this method returns, kvindex must refer to sufficient unused
     * storage to store one METADATA.
     */
    public synchronized void collect(K key, V value, final int partition
                                     ) throws IOException {
      reporter.progress();
      if (key.getClass() != keyClass) {
        throw new IOException("Type mismatch in key from map: expected "
                              + keyClass.getName() + ", received "
                              + key.getClass().getName());
      }
      if (value.getClass() != valClass) {
        throw new IOException("Type mismatch in value from map: expected "
                              + valClass.getName() + ", received "
                              + value.getClass().getName());
      }
      if (partition < 0 || partition >= partitions) {
        throw new IOException("Illegal partition for " + key + " (" +
            partition + ")");
      }
      checkSpillException();
      bufferRemaining -= METASIZE;
      if (bufferRemaining <= 0) {
        // start spill if the thread is not running and the soft limit has been
        // reached
        spillLock.lock();
        try {
          do {
            if (!spillInProgress) {
              final int kvbidx = 4 * kvindex;
              final int kvbend = 4 * kvend;

View on GitHub (pinned to 2add963021)

Solutions

Call job.setMapOutputValueClass(...) with exactly the class map() writes (and setMapOutputKeyClass for the key)
Check the Mapper<K,V,K2,V2> generic signature and every context.write(...) site
Verify combiner input value types equal the map output value types
Run a one-record local integration test of the mapper before submitting

Example fix

// before
public class CountMapper extends Mapper<LongWritable, Text, Text, LongWritable> { /* writes LongWritable values */ }
job.setOutputValueClass(Text.class);        // only final output classes set

// after
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(LongWritable.class);  // what the mapper emits
job.setOutputKeyClass(Text.class);          // what the reducer emits
job.setOutputValueClass(Text.class);

Defensive patterns

Strategy: validation

Validate before calling

// before job.submit(): assert the exact value class the mapper emits
Class<?> emitted = IntWritable.class; // class constructed at every context.write(...) in map()
if (!job.getMapOutputValueClass().equals(emitted)) {
  throw new IllegalStateException("Map output value class should be " + emitted.getName()
      + ", configured " + job.getMapOutputValueClass().getName());
}

Prevention

Set job.setMapOutputValueClass(...) explicitly in every job whose intermediate value type differs from the output type
Never emit subclasses of the declared Writable - getClass() is an exact match
Include a combiner round-trip test when intermediate types changed

When it happens

Trigger: Mapper emits value instances of a class different from job.getMapOutputValueClass(); setMapOutputValueClass never called so the final output value class is assumed; old-API JobConf with a wrong mapred.mapoutput.valueclass; emitting a subclass (e.g., a custom Writable extending Text).

Common situations: Mapper generic value parameter changed without updating the job; mapper writes IntWritable while only setOutputValueClass(Text.class) was set; combiner/reducer expectations drifting from the mapper; Avro/Pig-generated jobs with mis-set intermediate types.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/faf72ae144f6eb8d. Report an issue: GitHub.