apache/hadoop · error · IOException

Type mismatch in value from map: expected

Error message

Type mismatch in value from map: expected {valueClassName}, received {actualValueClassName}

What it means

The sorting collector type-checks every value passed to OutputCollector.collect() / Context.write() with an exact getClass() comparison against the configured map-output value class (mapreduce.map.output.value.class, set via Job.setMapOutputValueClass). When map() emits a value whose runtime class - including a subclass - differs from the configured one, the map task fails with this IOException on the first offending record.

Solutions

Call job.setMapOutputValueClass(...) with exactly the class map() writes (and setMapOutputKeyClass for the key)
Check the Mapper<K,V,K2,V2> generic signature and every context.write(...) site
Verify combiner input value types equal the map output value types
Run a one-record local integration test of the mapper before submitting

Example fix

// before
public class CountMapper extends Mapper<LongWritable, Text, Text, LongWritable> { /* writes LongWritable values */ }
job.setOutputValueClass(Text.class);        // only final output classes set

// after
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(LongWritable.class);  // what the mapper emits
job.setOutputKeyClass(Text.class);          // what the reducer emits
job.setOutputValueClass(Text.class);

Defensive patterns

Strategy: validation

Validate before calling

// before job.submit(): assert the exact value class the mapper emits
Class<?> emitted = IntWritable.class; // class constructed at every context.write(...) in map()
if (!job.getMapOutputValueClass().equals(emitted)) {
  throw new IllegalStateException("Map output value class should be " + emitted.getName()
      + ", configured " + job.getMapOutputValueClass().getName());
}

Prevention

Set job.setMapOutputValueClass(...) explicitly in every job whose intermediate value type differs from the output type
Never emit subclasses of the declared Writable - getClass() is an exact match
Include a combiner round-trip test when intermediate types changed

When it happens

Trigger: Mapper emits value instances of a class different from job.getMapOutputValueClass(); setMapOutputValueClass never called so the final output value class is assumed; old-API JobConf with a wrong mapred.mapoutput.valueclass; emitting a subclass (e.g., a custom Writable extending Text).

Common situations: Mapper generic value parameter changed without updating the job; mapper writes IntWritable while only setOutputValueClass(Text.class) was set; combiner/reducer expectations drifting from the mapper; Avro/Pig-generated jobs with mis-set intermediate types.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/faf72ae144f6eb8d. Report an issue: GitHub.

Appendix: source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java:1102

            sortSpillException);
      }
    }

    /**
     * Serialize the key, value to intermediate storage.
     * When this method returns, kvindex must refer to sufficient unused
     * storage to store one METADATA.
     */
    public synchronized void collect(K key, V value, final int partition
                                     ) throws IOException {
      reporter.progress();
      if (key.getClass() != keyClass) {
        throw new IOException("Type mismatch in key from map: expected "
                              + keyClass.getName() + ", received "
                              + key.getClass().getName());
      }
      if (value.getClass() != valClass) {
        throw new IOException("Type mismatch in value from map: expected "
                              + valClass.getName() + ", received "
                              + value.getClass().getName());
      }
      if (partition < 0 || partition >= partitions) {
        throw new IOException("Illegal partition for " + key + " (" +
            partition + ")");
      }
      checkSpillException();
      bufferRemaining -= METASIZE;
      if (bufferRemaining <= 0) {
        // start spill if the thread is not running and the soft limit has been
        // reached
        spillLock.lock();
        try {
          do {
            if (!spillInProgress) {
              final int kvbidx = 4 * kvindex;
              final int kvbend = 4 * kvend;

View on GitHub (pinned to 2add963021)