apache/hadoop · error · IOException

Only 1 or 2 algorithm version is supported

Error message

Only 1 or 2 algorithm version is supported

What it means

FileOutputCommitter's constructor (FileOutputCommitter.java:134-145) reads mapreduce.fileoutputcommitter.algorithm.version (default 2) and only accepts 1 or 2. Any other value — including 0, 3, or a non-numeric string parsed oddly — throws IOException('Only 1 or 2 algorithm version is supported') when the committer is instantiated on the task or ApplicationMaster.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java:144

  
  /**
   * Create a file output committer
   * @param outputPath the job's output path, or null if you want the output
   * committer to act as a noop.
   * @param context the task's context
   * @throws IOException
   */
  @Private
  public FileOutputCommitter(Path outputPath, 
                             JobContext context) throws IOException {
    super(outputPath, context);
    Configuration conf = context.getConfiguration();
    algorithmVersion =
        conf.getInt(FILEOUTPUTCOMMITTER_ALGORITHM_VERSION,
                    FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT);
    LOG.info("File Output Committer Algorithm version is " + algorithmVersion);
    if (algorithmVersion != 1 && algorithmVersion != 2) {
      throw new IOException("Only 1 or 2 algorithm version is supported");
    }

    // if skip cleanup
    skipCleanup = conf.getBoolean(
        FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED,
        FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT);

    // if ignore failures in cleanup
    ignoreCleanupFailures = conf.getBoolean(
        FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED,
        FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT);

    LOG.info("FileOutputCommitter skip cleanup _temporary folders under " +
        "output directory:" + skipCleanup + ", ignore cleanup failures: " +
        ignoreCleanupFailures);

    if (algorithmVersion == 1 && skipCleanup) {
        LOG.warn("Skip cleaning up when using FileOutputCommitter V1 can lead to unexpected behaviors. " +

View on GitHub (pinned to 2add963021)

Solutions

1. Set the property to 1 or 2 (or remove it to use the default 2): <property><name>mapreduce.fileoutputcommitter.algorithm.version</name><value>2</value></property> in mapred-site.xml or -Dmapreduce.fileoutputcommitter.algorithm.version=1
2. Search all config layers (job code, -D flags, client conf dir, cluster mapred-site.xml, schedulers/tool wrappers) for the property and correct every occurrence
3. If you intended 'no commit', don't fake it with version 0 — use NullOutputFormat or direct-to-output writes via a custom committer
4. For S3/object stores prefer the S3A committer magic/output committers rather than algorithm.version tweaks

Example fix

# before
hadoop jar app.jar Driver -Dmapreduce.fileoutputcommitter.algorithm.version=0 /in /out
# -> IOException: Only 1 or 2 algorithm version is supported

# after
hadoop jar app.jar Driver -Dmapreduce.fileoutputcommitter.algorithm.version=1 /in /out

static void requireSupportedCommitterVersion(Configuration conf) {
  int v = conf.getInt("mapreduce.fileoutputcommitter.algorithm.version", 2);
  if (v != 1 && v != 2)
    throw new IllegalArgumentException("mapreduce.fileoutputcommitter.algorithm.version must be 1 or 2, got " + v);
}

try { new FileOutputCommitter(outPath, context); } catch (IOException e) { if (e.getMessage().contains("algorithm version")) throw new IllegalArgumentException("Fix mapreduce.fileoutputcommitter.algorithm.version to 1 or 2 in job/mapred-site config", e); throw e; }

When it happens

Trigger: Setting mapreduce.fileoutputcommitter.algorithm.version to something other than 1 or 2 in the job config, cluster-wide mapred-site.xml, or via -D at submit time (e.g. version 0 for 'no commit' or 3 by typo); config copied from documentation for a different Hadoop version; tools (Spark/SparkSQL, Pig, Hive, Cascading) that set the property expecting different valid ranges.

Common situations: Trying to disable commit-time rename with version 0 (some blog posts suggest it; unsupported here); typos like mapreduce.fileoutputcommitter.algorithm.version=2.0 causing getInt to fail differently or version=3; rolling upgrades where old jobs target a new cluster with stricter validation; explicit version pinning after reading about v1-vs-v2 data-loss debates.

Related errors

• Improper queue name : {nameValue}
• Failed to create {clazz}:{e}
• Unable to recover task %s, output: %s
• value cannot be blank
• ${value} is not in expected format.Expected format is <numbe

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ab0fc207e30371fe. Report an issue: GitHub.