apache/hadoop · error · IOException

Cannot recover task {taskAttemptID}

Error message

Cannot recover task {taskAttemptID}

What it means

The Manifest Committer does not implement task output recovery; its recoverTask() logs 'Rejecting recoverTask(<id>)' at WARN and unconditionally throws IOException('Cannot recover task <taskAttemptID>'). It fires when the MapReduce ApplicationMaster restarts with job recovery enabled and the framework asks each committer to recover the previous attempt's committed task output.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/committer/manifest/ManifestCommitter.java:290

   */
  @Override
  public boolean isRecoverySupported(final JobContext jobContext)
      throws IOException {
    LOG.info("Probe for isRecoverySupported({}): returning false",
        jobContext.getJobID());
    return false;
  }

  /**
   *
   * @param taskContext Context of the task whose output is being recovered
   * @throws IOException always
   */
  @Override
  public void recoverTask(final TaskAttemptContext taskContext)
      throws IOException {
    LOG.warn("Rejecting recoverTask({}) call", taskContext.getTaskAttemptID());
    throw new IOException("Cannot recover task "
        + taskContext.getTaskAttemptID());
  }

  /**
   * Commit the task.
   * This is where the task attempt tree list takes place.
   * @param context task context.
   * @throws IOException IO Failure.
   */
  @Override
  public void commitTask(final TaskAttemptContext context)
      throws IOException {
    ManifestCommitterConfig committerConfig = enterCommitter(true,
        context);
    try {
      StageConfig stageConfig = committerConfig.createStageConfig()
          .withOperations(createManifestStoreOperations())
          .build();

View on GitHub (pinned to 2add963021)

Solutions

  1. Disable output committer recovery for these jobs: set mapreduce.outputcommitter.recovery.enabled=false (the default).
  2. If recovery matters more than the committer, switch back to a committer that supports it (FileOutputCommitter).
  3. If already mid-failure: accept the job as failed and resubmit into a clean output path - there is no way to force the manifest committer to recover.
  4. Check the WARN 'Rejecting recoverTask' line in AM logs to confirm this is the throw you hit.

Example fix

// before: recovery enabled alongside manifest committer
conf.setBoolean("mapreduce.outputcommitter.recovery.enabled", true);

// after: manifest committer rejects recovery by design
conf.setBoolean("mapreduce.outputcommitter.recovery.enabled", false);
Defensive patterns

Strategy: validation

Validate before calling

// guard the incompatible combination before submit
String factory = conf.get("mapreduce.outputcommitter.factory.scheme." + scheme, "");
if (factory.contains("ManifestCommitterFactory")
    && conf.getBoolean("mapreduce.outputcommitter.recovery.enabled", false)) {
  throw new IllegalArgumentException(
      "manifest committer does not support task recovery; disable recovery or change committer");
}

Try / catch

try {
  job.waitForCompletion(true);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Cannot recover task")) {
    // unrecoverable by design: disable mapreduce.outputcommitter.recovery.enabled and resubmit
  }
}

Prevention

When it happens

Trigger: A job using the manifest committer (e.g. bound via mapreduce.outputcommitter.factory.scheme.abfs) whose AM is restarted (RM reschedule, node loss) while MR job/committer recovery is enabled, so OutputCommitter.recoverTask() is invoked during the job restart path.

Common situations: Migrating jobs from FileOutputCommitter (whose algorithm v1 supports recovery) to the manifest committer on ABFS/GCS while keeping recovery settings; cluster defaults or platform configs enabling output committer recovery after an upgrade.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/389abbe043208cb8. Report an issue: GitHub.