{"record":{"id":"389abbe043208cb8","repo":"apache/hadoop","slug":"cannot-recover-task-taskattemptid","errorCode":null,"errorMessage":"Cannot recover task {taskAttemptID}","messagePattern":"Cannot recover task (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/ManifestCommitter.java","lineNumber":290,"sourceCode":"   */\n  @Override\n  public boolean isRecoverySupported(final JobContext jobContext)\n      throws IOException {\n    LOG.info(\"Probe for isRecoverySupported({}): returning false\",\n        jobContext.getJobID());\n    return false;\n  }\n\n  /**\n   *\n   * @param taskContext Context of the task whose output is being recovered\n   * @throws IOException always\n   */\n  @Override\n  public void recoverTask(final TaskAttemptContext taskContext)\n      throws IOException {\n    LOG.warn(\"Rejecting recoverTask({}) call\", taskContext.getTaskAttemptID());\n    throw new IOException(\"Cannot recover task \"\n        + taskContext.getTaskAttemptID());\n  }\n\n  /**\n   * Commit the task.\n   * This is where the task attempt tree list takes place.\n   * @param context task context.\n   * @throws IOException IO Failure.\n   */\n  @Override\n  public void commitTask(final TaskAttemptContext context)\n      throws IOException {\n    ManifestCommitterConfig committerConfig = enterCommitter(true,\n        context);\n    try {\n      StageConfig stageConfig = committerConfig.createStageConfig()\n          .withOperations(createManifestStoreOperations())\n          .build();","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/ManifestCommitter.java#L272-L308","documentation":"The Manifest Committer does not implement task output recovery; its recoverTask() logs 'Rejecting recoverTask(<id>)' at WARN and unconditionally throws IOException('Cannot recover task <taskAttemptID>'). It fires when the MapReduce ApplicationMaster restarts with job recovery enabled and the framework asks each committer to recover the previous attempt's committed task output.","triggerScenarios":"A job using the manifest committer (e.g. bound via mapreduce.outputcommitter.factory.scheme.abfs) whose AM is restarted (RM reschedule, node loss) while MR job/committer recovery is enabled, so OutputCommitter.recoverTask() is invoked during the job restart path.","commonSituations":"Migrating jobs from FileOutputCommitter (whose algorithm v1 supports recovery) to the manifest committer on ABFS/GCS while keeping recovery settings; cluster defaults or platform configs enabling output committer recovery after an upgrade.","solutions":["Disable output committer recovery for these jobs: set mapreduce.outputcommitter.recovery.enabled=false (the default).","If recovery matters more than the committer, switch back to a committer that supports it (FileOutputCommitter).","If already mid-failure: accept the job as failed and resubmit into a clean output path - there is no way to force the manifest committer to recover.","Check the WARN 'Rejecting recoverTask' line in AM logs to confirm this is the throw you hit."],"exampleFix":"// before: recovery enabled alongside manifest committer\nconf.setBoolean(\"mapreduce.outputcommitter.recovery.enabled\", true);\n\n// after: manifest committer rejects recovery by design\nconf.setBoolean(\"mapreduce.outputcommitter.recovery.enabled\", false);","handlingStrategy":"validation","validationCode":"// guard the incompatible combination before submit\nString factory = conf.get(\"mapreduce.outputcommitter.factory.scheme.\" + scheme, \"\");\nif (factory.contains(\"ManifestCommitterFactory\")\n    && conf.getBoolean(\"mapreduce.outputcommitter.recovery.enabled\", false)) {\n  throw new IllegalArgumentException(\n      \"manifest committer does not support task recovery; disable recovery or change committer\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  job.waitForCompletion(true);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Cannot recover task\")) {\n    // unrecoverable by design: disable mapreduce.outputcommitter.recovery.enabled and resubmit\n  }\n}","preventionTips":["Keep mapreduce.outputcommitter.recovery.enabled=false (default) when using the manifest committer.","After AM restarts, expect manifest-committer jobs to fail rather than resume - budget for a clean resubmit.","Document the recovery/committer matrix in job onboarding docs."],"tags":["hadoop","mapreduce","manifest-committer","job-recovery","am-restart"],"backgroundTag":"task-recovery-unsupported","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}