apache/beam · error · IllegalStateException

Unable to abort task

Error message

Unable to abort task %s of job %s

What it means

HadoopFormatIO's sink task wrapper calls abortTask() when a task fails, delegating to the OutputCommitter. If abortTask itself throws an IOException, the error is converted to this IllegalStateException — notably the original cause is not chained, so the message only identifies task and job ids.

Solutions

1. Check the earlier IOException in the logs — this error masks the real abort failure cause
2. Verify the output filesystem is reachable and the committer's temp paths exist/are writable
3. Use a standard OutputCommitter (e.g., FileOutputCommitter) compatible with the Hadoop version

Example fix

// before
throw new IllegalStateException(String.format("Unable to abort task %s of job %s", getTaskId(), getJobId()));
// after
throw new IllegalStateException(String.format("Unable to abort task %s of job %s", getTaskId(), getJobId()), e); // chain cause

try {
  writeResult();
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Unable to abort task")) {
    LOG.error("Cleanup failed; check filesystem availability and output committer config", e);
    throw e;
  }
  throw e;
}

When it happens

Trigger: Failure cleanup path: the task failed, and OutputCommitter.abortTask(taskAttemptContext) threw an IOException (e.g., filesystem unavailability while deleting temporary output).

Common situations: HDFS/cluster unavailability during job cleanup; misconfigured OutputCommitter that cannot abort; temporary-output directories already removed or permission issues.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Unable to create RecordWriter object:
• AUTO is not supported for writing
• Cannot create reader as source is not split yet.
• Cannot provide because does not implement the interface
• Channel is closed

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/dd9b6b9a792ef70f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java:1557

    }

    TaskAttemptContext getTaskAttemptContext() {
      return taskAttemptContext;
    }

    int getTaskId() {
      return taskAttemptContext.getTaskAttemptID().getTaskID().getId();
    }

    String getJobId() {
      return taskAttemptContext.getJobID().getJtIdentifier();
    }

    void abortTask() {
      try {
        outputCommitter.abortTask(taskAttemptContext);
      } catch (IOException e) {
        throw new IllegalStateException(
            String.format("Unable to abort task %s of job %s", getTaskId(), getJobId()));
      }
    }

    private RecordWriter<KeyT, ValueT> initRecordWriter(
        OutputFormat<KeyT, ValueT> outputFormatObj, TaskAttemptContext taskAttemptContext)
        throws IllegalStateException {
      try {
        LOG.info(
            "Creating new RecordWriter for task {} of Job with id {}.",
            taskAttemptContext.getTaskAttemptID().getTaskID().getId(),
            taskAttemptContext.getJobID().getJtIdentifier());
        return outputFormatObj.getRecordWriter(taskAttemptContext);
      } catch (InterruptedException | IOException e) {
        throw new IllegalStateException("Unable to create RecordWriter object: ", e);
      }
    }

View on GitHub (pinned to 12126d8942)