apache/hadoop · error · IllegalArgumentException

The Reducer output key class does not match the Mapper input

Error message

The Reducer output key class does not match the Mapper input key class

What it means

When the first reducer-side mapper is added (index 0), Chain validates type continuity of the pipeline: the reducer's configured output key/value classes must be assignable to the new mapper's input key/value classes. validateKeyValueTypes throws IllegalArgumentException("The Reducer output key class does not match the Mapper input key class") (and the analogous value message) when inputKeyClass.isAssignableFrom(reducer output key class) fails.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/chain/Chain.java:679

          throw new IllegalStateException("Reducer has been already set");
        }
      }
    }
  }

  protected static void validateKeyValueTypes(boolean isMap,
      Configuration jobConf, Class<?> inputKeyClass, Class<?> inputValueClass,
      Class<?> outputKeyClass, Class<?> outputValueClass, int index,
      String prefix) {
    // if it is a reducer chain and the first Mapper is being added check the
    // key and value input classes of the mapper match those of the reducer
    // output.
    if (!isMap && index == 0) {
      Configuration reducerConf = getChainElementConf(jobConf, prefix
          + CHAIN_REDUCER_CONFIG);
      if (!inputKeyClass.isAssignableFrom(reducerConf.getClass(
          REDUCER_OUTPUT_KEY_CLASS, null))) {
        throw new IllegalArgumentException("The Reducer output key class does"
            + " not match the Mapper input key class");
      }
      if (!inputValueClass.isAssignableFrom(reducerConf.getClass(
          REDUCER_OUTPUT_VALUE_CLASS, null))) {
        throw new IllegalArgumentException("The Reducer output value class"
            + " does not match the Mapper input value class");
      }
    } else if (index > 0) {
      // check the that the new Mapper in the chain key and value input classes
      // match those of the previous Mapper output.
      Configuration previousMapperConf = getChainElementConf(jobConf, prefix
          + CHAIN_MAPPER_CONFIG + (index - 1));
      if (!inputKeyClass.isAssignableFrom(previousMapperConf.getClass(
          MAPPER_OUTPUT_KEY_CLASS, null))) {
        throw new IllegalArgumentException("The specified Mapper input key class does"
            + " not match the previous Mapper's output key class.");
      }
      if (!inputValueClass.isAssignableFrom(previousMapperConf.getClass(

View on GitHub (pinned to 2add963021)

Solutions

Make the first reducer-side mapper's input key/value classes exactly the reducer's output key/value classes (or supertypes).
Re-check the parameter order of both calls: setReducer(job, reducer, inK, inV, outK, outV, byValue) and addMapper(job, mapper, inK, inV, outK, outV, byValue).
Add a wiring unit test that calls setReducer + addMapper against a test Configuration so mismatches fail in CI, not at job submission.
Update adjacent chain steps after any mapper/reducer signature change (every adjacent pair must agree).

Example fix

// before: reducer outputs Text, mapper claims LongWritable input
ChainReducer.setReducer(job, Reduce.class,
    LongWritable.class, Text.class, Text.class, IntWritable.class, false);
ChainReducer.addMapper(job, PostMap.class,
    LongWritable.class, IntWritable.class, Text.class, Text.class, false); // throws

// after: first mapper input matches reducer output
ChainReducer.addMapper(job, PostMap.class,
    Text.class, IntWritable.class, Text.class, Text.class, false);

Defensive patterns

Strategy: type-guard

Validate before calling

static void checkChainTypes(Class<?> reducerOutK, Class<?> reducerOutV,
                                 Class<?> mapperInK, Class<?> mapperInV) {
  if (!mapperInK.isAssignableFrom(reducerOutK) || !mapperInV.isAssignableFrom(reducerOutV)) {
    throw new IllegalArgumentException("First mapper input [" + mapperInK + "," + mapperInV
        + "] must match reducer output [" + reducerOutK + "," + reducerOutV + "]");
  }
}

Type guard

boolean chainTypesCompatible(Class<?> reducerOutK, Class<?> reducerOutV,
                               Class<?> mapperInK, Class<?> mapperInV) {
  return mapperInK.isAssignableFrom(reducerOutK)
      && mapperInV.isAssignableFrom(reducerOutV);
}

Prevention

Make the first reducer-side mapper's input classes identical to the reducer's output classes.
Memorize the parameter order (inK, inV, outK, outV) for both setReducer and addMapper; wrong order is the top cause.
Add a CI wiring test for every chain; update adjacent steps whenever a mapper/reducer signature changes.

When it happens

Trigger: ChainReducer.setReducer(job, R.class, kIn, vIn, Text.class, IntWritable.class, ...) followed by ChainReducer.addMapper(job, M.class, LongWritable.class, ...) - the first mapper's declared input key (LongWritable) does not match the reducer's declared output key (Text). Also triggered by swapping the four class arguments of either call.

Common situations: Type drift after refactoring mapper/reducer signatures; mis-ordering the (inputKey, inputValue, outputKey, outputValue) parameters, which are easy to mix up since they are all Class tokens checked only at wiring time.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4f3c0c00a29228fa. Report an issue: GitHub.