{"record":{"id":"ab0fc207e30371fe","repo":"apache/hadoop","slug":"only-1-or-2-algorithm-version-is-supported","errorCode":null,"errorMessage":"Only 1 or 2 algorithm version is supported","messagePattern":"Only 1 or 2 algorithm version is supported","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java","lineNumber":144,"sourceCode":"  \n  /**\n   * Create a file output committer\n   * @param outputPath the job's output path, or null if you want the output\n   * committer to act as a noop.\n   * @param context the task's context\n   * @throws IOException\n   */\n  @Private\n  public FileOutputCommitter(Path outputPath, \n                             JobContext context) throws IOException {\n    super(outputPath, context);\n    Configuration conf = context.getConfiguration();\n    algorithmVersion =\n        conf.getInt(FILEOUTPUTCOMMITTER_ALGORITHM_VERSION,\n                    FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT);\n    LOG.info(\"File Output Committer Algorithm version is \" + algorithmVersion);\n    if (algorithmVersion != 1 && algorithmVersion != 2) {\n      throw new IOException(\"Only 1 or 2 algorithm version is supported\");\n    }\n\n    // if skip cleanup\n    skipCleanup = conf.getBoolean(\n        FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED,\n        FILEOUTPUTCOMMITTER_CLEANUP_SKIPPED_DEFAULT);\n\n    // if ignore failures in cleanup\n    ignoreCleanupFailures = conf.getBoolean(\n        FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED,\n        FILEOUTPUTCOMMITTER_CLEANUP_FAILURES_IGNORED_DEFAULT);\n\n    LOG.info(\"FileOutputCommitter skip cleanup _temporary folders under \" +\n        \"output directory:\" + skipCleanup + \", ignore cleanup failures: \" +\n        ignoreCleanupFailures);\n\n    if (algorithmVersion == 1 && skipCleanup) {\n        LOG.warn(\"Skip cleaning up when using FileOutputCommitter V1 can lead to unexpected behaviors. \" +","sourceCodeStart":126,"sourceCodeEnd":162,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L126-L162","documentation":"FileOutputCommitter's constructor (FileOutputCommitter.java:134-145) reads mapreduce.fileoutputcommitter.algorithm.version (default 2) and only accepts 1 or 2. Any other value — including 0, 3, or a non-numeric string parsed oddly — throws IOException('Only 1 or 2 algorithm version is supported') when the committer is instantiated on the task or ApplicationMaster.","triggerScenarios":"Setting mapreduce.fileoutputcommitter.algorithm.version to something other than 1 or 2 in the job config, cluster-wide mapred-site.xml, or via -D at submit time (e.g. version 0 for 'no commit' or 3 by typo); config copied from documentation for a different Hadoop version; tools (Spark/SparkSQL, Pig, Hive, Cascading) that set the property expecting different valid ranges.","commonSituations":"Trying to disable commit-time rename with version 0 (some blog posts suggest it; unsupported here); typos like mapreduce.fileoutputcommitter.algorithm.version=2.0 causing getInt to fail differently or version=3; rolling upgrades where old jobs target a new cluster with stricter validation; explicit version pinning after reading about v1-vs-v2 data-loss debates.","solutions":["Set the property to 1 or 2 (or remove it to use the default 2): <property><name>mapreduce.fileoutputcommitter.algorithm.version</name><value>2</value></property> in mapred-site.xml or -Dmapreduce.fileoutputcommitter.algorithm.version=1","Search all config layers (job code, -D flags, client conf dir, cluster mapred-site.xml, schedulers/tool wrappers) for the property and correct every occurrence","If you intended 'no commit', don't fake it with version 0 — use NullOutputFormat or direct-to-output writes via a custom committer","For S3/object stores prefer the S3A committer magic/output committers rather than algorithm.version tweaks"],"exampleFix":"# before\nhadoop jar app.jar Driver -Dmapreduce.fileoutputcommitter.algorithm.version=0 /in /out\n# -> IOException: Only 1 or 2 algorithm version is supported\n\n# after\nhadoop jar app.jar Driver -Dmapreduce.fileoutputcommitter.algorithm.version=1 /in /out","handlingStrategy":"validation","validationCode":"static void requireSupportedCommitterVersion(Configuration conf) {\n  int v = conf.getInt(\"mapreduce.fileoutputcommitter.algorithm.version\", 2);\n  if (v != 1 && v != 2)\n    throw new IllegalArgumentException(\"mapreduce.fileoutputcommitter.algorithm.version must be 1 or 2, got \" + v);\n}","typeGuard":null,"tryCatchPattern":"try { new FileOutputCommitter(outPath, context); } catch (IOException e) { if (e.getMessage().contains(\"algorithm version\")) throw new IllegalArgumentException(\"Fix mapreduce.fileoutputcommitter.algorithm.version to 1 or 2 in job/mapred-site config\", e); throw e; }","preventionTips":["Only use 1 or 2 for mapreduce.fileoutputcommitter.algorithm.version","Lint cluster mapred-site.xml and submit-time -D flags for this property during upgrades","Remember the default is 2; explicitly pin it only when you need v1 semantics"],"tags":["hadoop","mapreduce","output-committer","configuration","validation"],"backgroundTag":"invalid-config-value","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}