{"record":{"id":"46ca891665728157","repo":"apache/hadoop","slug":"cannot-recover-task-output-for-first-attempt","errorCode":null,"errorMessage":"Cannot recover task output for first attempt...","messagePattern":"Cannot recover task output for first attempt\\.\\.\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java","lineNumber":703,"sourceCode":"  @Deprecated\n  public boolean isRecoverySupported() {\n    return true;\n  }\n\n  @Override\n  public boolean isCommitJobRepeatable(JobContext context) throws IOException {\n    return algorithmVersion == 2;\n  }\n\n  @Override\n  public void recoverTask(TaskAttemptContext context)\n      throws IOException {\n    if(hasOutputPath()) {\n      context.progress();\n      TaskAttemptID attemptId = context.getTaskAttemptID();\n      int previousAttempt = getAppAttemptId(context) - 1;\n      if (previousAttempt < 0) {\n        throw new IOException (\"Cannot recover task output for first attempt...\");\n      }\n\n      Path previousCommittedTaskPath = getCommittedTaskPath(\n          previousAttempt, context);\n      FileSystem fs = previousCommittedTaskPath.getFileSystem(context.getConfiguration());\n      if (LOG.isDebugEnabled()) {\n        LOG.debug(\"Trying to recover task from \" + previousCommittedTaskPath);\n      }\n      if (algorithmVersion == 1) {\n        if (fs.exists(previousCommittedTaskPath)) {\n          Path committedTaskPath = getCommittedTaskPath(context);\n          if (!fs.delete(committedTaskPath, true) &&\n              fs.exists(committedTaskPath)) {\n            throw new IOException(\"Could not delete \" + committedTaskPath);\n          }\n          //Rename can fail if the parent directory does not yet exist.\n          Path committedParent = committedTaskPath.getParent();\n          fs.mkdirs(committedParent);","sourceCodeStart":685,"sourceCodeEnd":721,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java#L685-L721","documentation":"Thrown from FileOutputCommitter.recoverTask (FileOutputCommitter.java:703) when getAppAttemptId(context) - 1 < 0, i.e. the task-attempt context reports application attempt id 0. recoverTask exists to move output committed by a PREVIOUS application attempt into the current attempt's committed-task path; for the first attempt there is nothing to recover, so the call is a misuse of the API.","triggerScenarios":"Calling recoverTask(context) where context.getTaskAttemptID().getId() == 0. Happens in unit tests that invoke recoverTask on a freshly constructed TaskAttemptContext, in custom committers/OutputCommitter wrappers that always call recoverTask, or when MRAppMaster recovery code paths run against a first-attempt context.","commonSituations":"Developers writing unit tests for custom OutputCommitters and delegating to FileOutputCommitter.recoverTask; tooling that manually drives the committer lifecycle (setupJob -> recoverTask -> commitTask) without bumping the attempt id.","solutions":["Only call recoverTask for application attempts >= 1; guard with the attempt id before delegating","In tests, construct the TaskAttemptID with an attempt number >= 1 (new TaskAttemptID(..., 1)) when exercising recovery","If you wrap FileOutputCommitter, forward recoverTask only when the framework actually signals a restart (job is in recovery state), which it never does for attempt 0"],"exampleFix":"// before\npublic void recoverTask(TaskAttemptContext context) throws IOException {\n  committer.recoverTask(context); // throws for first attempt (id 0)\n}\n\n// after\npublic void recoverTask(TaskAttemptContext context) throws IOException {\n  if (context.getTaskAttemptID().getId() > 0) {\n    committer.recoverTask(context);\n  }\n}","handlingStrategy":"validation","validationCode":"// guard before delegating recovery\nint appAttempt = context.getTaskAttemptID().getId();\nif (appAttempt > 0) {\n  committer.recoverTask(context);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat recoverTask as framework-internal; call it only for attempts >= 1","In committer unit tests, build TaskAttemptID with attempt number >= 1"],"tags":["hadoop","mapreduce","file-output-committer","task-recovery","app-attempt","api-misuse"],"backgroundTag":"mapreduce-task-recovery-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}