apache/hadoop · error · IllegalArgumentException

Named output '{}' already alreadyDefined

Error message

Named output '{}' already alreadyDefined

What it means

MultipleOutputs stores named outputs as entries under the 'mo.namedOutputs' configuration property. checkNamedOutput(conf, name, true) runs inside addNamedOutput/addMultiNamedOutput and throws this IllegalArgumentException (the message contains a duplicated word, 'already alreadyDefined') when the name is already present in that list. It is a guard against re-registering the same output channel twice with potentially different settings.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/MultipleOutputs.java:176

  private static final String COUNTERS_GROUP = MultipleOutputs.class.getName();
  private static final Logger LOG = LoggerFactory.getLogger(MultipleOutputs.class);

  /**
   * Checks if a named output is alreadyDefined or not.
   *
   * @param conf           job conf
   * @param namedOutput    named output names
   * @param alreadyDefined whether the existence/non-existence of
   *                       the named output is to be checked
   * @throws IllegalArgumentException if the output name is alreadyDefined or
   *                                  not depending on the value of the
   *                                  'alreadyDefined' parameter
   */
  private static void checkNamedOutput(JobConf conf, String namedOutput,
                                       boolean alreadyDefined) {
    List<String> definedChannels = getNamedOutputsList(conf);
    if (alreadyDefined && definedChannels.contains(namedOutput)) {
      throw new IllegalArgumentException("Named output '" + namedOutput +
        "' already alreadyDefined");
    } else if (!alreadyDefined && !definedChannels.contains(namedOutput)) {
      throw new IllegalArgumentException("Named output '" + namedOutput +
        "' not defined");
    }
  }

  /**
   * Checks if a named output name is valid token.
   *
   * @param namedOutput named output Name
   * @throws IllegalArgumentException if the output name is not valid.
   */
  private static void checkTokenName(String namedOutput) {
    if (namedOutput == null || namedOutput.length() == 0) {
      throw new IllegalArgumentException(
        "Name cannot be NULL or emtpy");
    }

View on GitHub (pinned to 2add963021)

Solutions

1. Register each named output exactly once per JobConf; move the addNamedOutput call out of loops
2. Before adding, check MultipleOutputs.getNamedOutputsList(conf).contains(name) and skip if present
3. If two channels were intended, give them distinct names (e.g. text1, text2)
4. Rebuild the JobConf from scratch instead of reusing one that already has mo.namedOutputs set

Example fix

// before: second registration -> IllegalArgumentException
MultipleOutputs.addNamedOutput(conf, "text", TextOutputFormat.class, LongWritable.class, Text.class);
MultipleOutputs.addNamedOutput(conf, "text", TextOutputFormat.class, LongWritable.class, Text.class);

// after: idempotent registration
if (!MultipleOutputs.getNamedOutputsList(conf).contains("text")) {
  MultipleOutputs.addNamedOutput(conf, "text", TextOutputFormat.class, LongWritable.class, Text.class);
}

// make registration idempotent
if (!MultipleOutputs.getNamedOutputsList(conf).contains("text")) {
  MultipleOutputs.addNamedOutput(conf, "text", TextOutputFormat.class, LongWritable.class, Text.class);
}

When it happens

Trigger: Two calls to MultipleOutputs.addNamedOutput(conf, "text", ...) (or one addNamedOutput plus one addMultiNamedOutput) with the same name in job setup code. Also happens when driver code runs in a loop (e.g. per input path) and adds a fixed named output on every iteration against the same JobConf.

Common situations: Copy-pasted job setup blocks; refactoring that merges two job builders that both register the same channel; loops that add a channel once per shard instead of once per job.

Related errors

• Named output '{namedOutput}' already alreadyDefined
• Named output '{}' not defined
• Named output '{namedOutput}' not defined
• Invalid specification for distributed-cache artifacts of typ
• Unable to parse '{}' as a URI, check the setting for mapredu

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3a5c5116d66a8f66. Report an issue: GitHub.