apache/hadoop · error · IOException

No committer defined in mapreduce.outputcommitter.named.clas

Error message

No committer defined in mapreduce.outputcommitter.named.classname

What it means

NamedCommitterFactory.loadCommitterClass reads mapreduce.outputcommitter.named.classname and treats an empty value as fatal: it throws IOException('No committer defined in mapreduce.outputcommitter.named.classname') before the class is ever loaded. It means the named factory was selected for the output filesystem scheme but no committer implementation was named for it to create.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/NamedCommitterFactory.java:74

        | InvocationTargetException e) {
      throw new IOException("Failed to create " + clazz
          + ":" + e, e);
    }
  }

  /**
   * Load the class named in {@link #NAMED_COMMITTER_CLASS}.
   * @param context job or task context
   * @return the committer class
   * @throws IOException if no committer was defined.
   */
  private Class<? extends PathOutputCommitter> loadCommitterClass(
      JobContext context) throws IOException {
    Preconditions.checkNotNull(context, "null context");
    Configuration conf = context.getConfiguration();
    String value = conf.get(NAMED_COMMITTER_CLASS, "");
    if (value.isEmpty()) {
      throw new IOException("No committer defined in " + NAMED_COMMITTER_CLASS);
    }
    return conf.getClass(NAMED_COMMITTER_CLASS,
        FileOutputCommitter.class, PathOutputCommitter.class);
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Set the property: conf.set("mapreduce.outputcommitter.named.classname", FileOutputCommitter.class.getName()) or your PathOutputCommitter implementation.
2. If you did not intend the named factory, remove the mapreduce.outputcommitter.factory.scheme.* binding so PathOutputCommitterFactory falls back to FileOutputCommitter.
3. Dump the final configuration on the failing node (conf.get on the task, or -D dump of job.xml) to confirm the property survives job submission.
4. Check for trailing whitespace/empty string values which the isEmpty() check also rejects.

Example fix

// before: factory selected, committer never named
conf.set("mapreduce.outputcommitter.factory.scheme.hdfs",
    "org.apache.hadoop.mapreduce.lib.output.NamedCommitterFactory");

// after: name the implementation the factory must create
conf.set("mapreduce.outputcommitter.factory.scheme.hdfs",
    "org.apache.hadoop.mapreduce.lib.output.NamedCommitterFactory");
conf.set("mapreduce.outputcommitter.named.classname",
    "org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter");

// fail fast on the client before submission
String factory = conf.get("mapreduce.outputcommitter.factory.scheme." + scheme);
if (factory != null && factory.contains("NamedCommitterFactory")) {
  String impl = conf.get("mapreduce.outputcommitter.named.classname", "");
  Preconditions.checkArgument(!impl.trim().isEmpty(),
      "named factory selected but mapreduce.outputcommitter.named.classname is empty");
}

try {
  job.waitForCompletion(true);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("No committer defined")) {
    // config error: set the classname property and resubmit, do not retry as-is
  }
}

When it happens

Trigger: Setting mapreduce.outputcommitter.factory.scheme.<scheme> (or the default factory key) to NamedCommitterFactory while mapreduce.outputcommitter.named.classname is unset or blank in the effective (final, after job.xml merge) configuration.

Common situations: Copy-pasting the factory binding from docs or another cluster without also setting the classname key; a job template that sets the factory per scheme (hdfs, abfs, gcs) but only defines the classname for some jobs; Hive/Spark/Impala overlaying conf that clears the property; typos in the property name so the real value never lands.

Related errors

• Unable to recover task %s, output: %s
• Invalid state of the job for cleanup. State found " + jobRun
• Only 1 or 2 algorithm version is supported
• Failed to delete {}
• Failed to rename {} to {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b0600b2fda706e4f. Report an issue: GitHub.