apache/hadoop · error · IOException

Cannot recover task output for first attempt...

Error message

Cannot recover task output for first attempt...

What it means

Thrown from FileOutputCommitter.recoverTask (FileOutputCommitter.java:703) when getAppAttemptId(context) - 1 < 0, i.e. the task-attempt context reports application attempt id 0. recoverTask exists to move output committed by a PREVIOUS application attempt into the current attempt's committed-task path; for the first attempt there is nothing to recover, so the call is a misuse of the API.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java:703

  @Deprecated
  public boolean isRecoverySupported() {
    return true;
  }

  @Override
  public boolean isCommitJobRepeatable(JobContext context) throws IOException {
    return algorithmVersion == 2;
  }

  @Override
  public void recoverTask(TaskAttemptContext context)
      throws IOException {
    if(hasOutputPath()) {
      context.progress();
      TaskAttemptID attemptId = context.getTaskAttemptID();
      int previousAttempt = getAppAttemptId(context) - 1;
      if (previousAttempt < 0) {
        throw new IOException ("Cannot recover task output for first attempt...");
      }

      Path previousCommittedTaskPath = getCommittedTaskPath(
          previousAttempt, context);
      FileSystem fs = previousCommittedTaskPath.getFileSystem(context.getConfiguration());
      if (LOG.isDebugEnabled()) {
        LOG.debug("Trying to recover task from " + previousCommittedTaskPath);
      }
      if (algorithmVersion == 1) {
        if (fs.exists(previousCommittedTaskPath)) {
          Path committedTaskPath = getCommittedTaskPath(context);
          if (!fs.delete(committedTaskPath, true) &&
              fs.exists(committedTaskPath)) {
            throw new IOException("Could not delete " + committedTaskPath);
          }
          //Rename can fail if the parent directory does not yet exist.
          Path committedParent = committedTaskPath.getParent();
          fs.mkdirs(committedParent);

View on GitHub (pinned to 2add963021)

Solutions

1. Only call recoverTask for application attempts >= 1; guard with the attempt id before delegating
2. In tests, construct the TaskAttemptID with an attempt number >= 1 (new TaskAttemptID(..., 1)) when exercising recovery
3. If you wrap FileOutputCommitter, forward recoverTask only when the framework actually signals a restart (job is in recovery state), which it never does for attempt 0

Example fix

// before
public void recoverTask(TaskAttemptContext context) throws IOException {
  committer.recoverTask(context); // throws for first attempt (id 0)
}

// after
public void recoverTask(TaskAttemptContext context) throws IOException {
  if (context.getTaskAttemptID().getId() > 0) {
    committer.recoverTask(context);
  }
}

// guard before delegating recovery
int appAttempt = context.getTaskAttemptID().getId();
if (appAttempt > 0) {
  committer.recoverTask(context);
}

When it happens

Trigger: Calling recoverTask(context) where context.getTaskAttemptID().getId() == 0. Happens in unit tests that invoke recoverTask on a freshly constructed TaskAttemptContext, in custom committers/OutputCommitter wrappers that always call recoverTask, or when MRAppMaster recovery code paths run against a first-attempt context.

Common situations: Developers writing unit tests for custom OutputCommitters and delegating to FileOutputCommitter.recoverTask; tooling that manually drives the committer lifecycle (setupJob -> recoverTask -> commitTask) without bumping the attempt id.

Related errors

• getShuffleFinishTime() not supported for MapTask
• setShuffleFinishTime() not supported for MapTask
• Name output '{}' has not been defined as multi
• Could not delete {}
• Could not rename {} to {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/46ca891665728157. Report an issue: GitHub.