apache/hadoop · error · IllegalArgumentException

Named output '{namedOutput}' already alreadyDefined

Error message

Named output '{namedOutput}' already alreadyDefined

What it means

Thrown as IllegalArgumentException from MultipleOutputs.checkNamedOutputName (MultipleOutputs.java:266, message contains the upstream typo 'already alreadyDefined') when addNamedOutput is called with a name already present in the space-separated mapreduce.multipleoutputs config list. Each channel must be registered exactly once; the check exists because a second conf.set would corrupt the format/key/value properties of the channel.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.java:266

  private static void checkBaseOutputPath(String outputPath) {
    if (outputPath.equals(FileOutputFormat.PART)) {
      throw new IllegalArgumentException("output name cannot be 'part'");
    }
  }
  
  /**
   * Checks if a named output name is valid.
   *
   * @param namedOutput named output Name
   * @throws IllegalArgumentException if the output name is not valid.
   */
  private static void checkNamedOutputName(JobContext job,
      String namedOutput, boolean alreadyDefined) {
    checkTokenName(namedOutput);
    checkBaseOutputPath(namedOutput);
    List<String> definedChannels = getNamedOutputsList(job);
    if (alreadyDefined && definedChannels.contains(namedOutput)) {
      throw new IllegalArgumentException("Named output '" + namedOutput +
        "' already alreadyDefined");
    } else if (!alreadyDefined && !definedChannels.contains(namedOutput)) {
      throw new IllegalArgumentException("Named output '" + namedOutput +
        "' not defined");
    }
  }

  // Returns list of channel names.
  private static List<String> getNamedOutputsList(JobContext job) {
    List<String> names = new ArrayList<String>();
    StringTokenizer st = new StringTokenizer(
      job.getConfiguration().get(MULTIPLE_OUTPUTS, ""), " ");
    while (st.hasMoreTokens()) {
      names.add(st.nextToken());
    }
    return names;
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Register each named output exactly once per Job; move addNamedOutput calls into a single setup method
2. Before adding, read conf.get("mapreduce.multipleoutputs", "") and skip names already present
3. Use distinct channel names per component (prefix by domain: 'etlText', 'auditText')

Example fix

// before: retried driver code registers twice
MultipleOutputs.addNamedOutput(job, "text", TextOutputFormat.class, LongWritable.class, Text.class);
MultipleOutputs.addNamedOutput(job, "text", TextOutputFormat.class, LongWritable.class, Text.class); // throws

// after: idempotent registration
String registered = job.getConfiguration().get("mapreduce.multipleoutputs", "");
if (!Arrays.asList(registered.split(" ")).contains("text")) {
  MultipleOutputs.addNamedOutput(job, "text", TextOutputFormat.class, LongWritable.class, Text.class);
}

// idempotent registration helper
static void addNamedOutputOnce(Job job, String name,
    Class<? extends OutputFormat> fmt, Class<?> k, Class<?> v) {
  String list = job.getConfiguration().get("mapreduce.multipleoutputs", "");
  if (!Arrays.asList(list.split(" ")).contains(name)) {
    MultipleOutputs.addNamedOutput(job, name, fmt, k, v);
  }
}

When it happens

Trigger: Calling MultipleOutputs.addNamedOutput(job, "text", ...) twice on the same Job/Configuration — e.g. a driver loop that re-adds channels on retry, shared Configuration objects in unit tests, or two components each registering the same well-known channel name.

Common situations: Test harnesses reusing one Configuration across test methods; job drivers that configure named outputs both statically and from parsed arguments; teams merging two jobs whose channel names clash.

Related errors

• Named output '{}' already alreadyDefined
• Named output '{namedOutput}' not defined
• Named output '{}' not defined
• Name cannot be NULL or emtpy
• Name cannot be have a '{ch}' char

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a76f143db415d193. Report an issue: GitHub.