apache/hadoop · error · IllegalStateException

Reducer has been already set

Error message

Reducer has been already set

What it means

The mirror rule of the ordering check: checkReducerAlreadySet(shouldSet=false) throws IllegalStateException("Reducer has been already set") when the reducer-side chain is being (re)configured while the 'chain reducer class' key is already present - in practice, when ChainReducer.setReducer(...) is called a second time on the same Job/Configuration.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/chain/Chain.java:661

        outputKeyClass, outputValueClass, index, prefix);

    setMapperConf(isMap, jobConf, inputKeyClass, inputValueClass,
        outputKeyClass, outputValueClass, mapperConf, index, prefix);
  }

  // if a reducer chain check the Reducer has been already set or not
  protected static void checkReducerAlreadySet(boolean isMap,
      Configuration jobConf, String prefix, boolean shouldSet) {
    if (!isMap) {
      if (shouldSet) {
        if (jobConf.getClass(prefix + CHAIN_REDUCER_CLASS, null) == null) {
          throw new IllegalStateException(
              "A Mapper can be added to the chain only after the Reducer has "
                  + "been set");
        }
      } else {
        if (jobConf.getClass(prefix + CHAIN_REDUCER_CLASS, null) != null) {
          throw new IllegalStateException("Reducer has been already set");
        }
      }
    }
  }

  protected static void validateKeyValueTypes(boolean isMap,
      Configuration jobConf, Class<?> inputKeyClass, Class<?> inputValueClass,
      Class<?> outputKeyClass, Class<?> outputValueClass, int index,
      String prefix) {
    // if it is a reducer chain and the first Mapper is being added check the
    // key and value input classes of the mapper match those of the reducer
    // output.
    if (!isMap && index == 0) {
      Configuration reducerConf = getChainElementConf(jobConf, prefix
          + CHAIN_REDUCER_CONFIG);
      if (!inputKeyClass.isAssignableFrom(reducerConf.getClass(
          REDUCER_OUTPUT_KEY_CLASS, null))) {
        throw new IllegalArgumentException("The Reducer output key class does"

View on GitHub (pinned to 2add963021)

Solutions

1. Call ChainReducer.setReducer exactly once per Job; build a fresh Job/Configuration for each submission attempt.
2. Clear the chain keys (or, better, start from a new Configuration) when re-assembling a job.
3. Move chain wiring out of loops/retry paths into single-execution setup code.

Example fix

// before: retry path reuses the same job conf
ChainReducer.setReducer(job, Reduce.class, ...);   // first attempt
ChainReducer.setReducer(job, Reduce2.class, ...);  // retry -> IllegalStateException

// after: fresh job per submission
Job job = Job.getInstance(new Configuration(conf));
ChainReducer.setReducer(job, Reduce2.class, ...);

// guard submission-retry paths
boolean alreadyWired = conf.get("chain.reducer.class") != null;
if (alreadyWired) {
  job = Job.getInstance(new Configuration(conf)); // fresh job, then setReducer once
}
ChainReducer.setReducer(job, Reduce.class, ...);

catch (IllegalStateException e) with message "Reducer has been already set": discard the Job and rebuild from a fresh Configuration instead of trying to overwrite the existing reducer.

When it happens

Trigger: Two ChainReducer.setReducer(...) calls on one Job/Configuration: submission-retry logic reusing a Configuration, job templates merged on top of an already-wired conf, or copy-pasted setup blocks.

Common situations: Job assembly code that 'reconfigures' a job for retry instead of building a fresh Job; combining config snippets from multiple sources that each set the chain reducer; loops that set the reducer per input.

Related errors

• A Mapper can be added to the chain only after the Reducer ha
• The Reducer output key class does not match the Mapper input
• Unable to recover task %s, output: %s
• Shutdown in progress, cannot add a shutdownHook
• Shutdown in progress, cannot remove a shutdownHook

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/05473a3318c07c73. Report an issue: GitHub.