{"record":{"id":"dd9b6b9a792ef70f","repo":"apache/beam","slug":"unable-to-abort-task-s-of-job-s","errorCode":null,"errorMessage":"Unable to abort task %s of job %s","messagePattern":"Unable to abort task (.+?) of job (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java","lineNumber":1557,"sourceCode":"    }\n\n    TaskAttemptContext getTaskAttemptContext() {\n      return taskAttemptContext;\n    }\n\n    int getTaskId() {\n      return taskAttemptContext.getTaskAttemptID().getTaskID().getId();\n    }\n\n    String getJobId() {\n      return taskAttemptContext.getJobID().getJtIdentifier();\n    }\n\n    void abortTask() {\n      try {\n        outputCommitter.abortTask(taskAttemptContext);\n      } catch (IOException e) {\n        throw new IllegalStateException(\n            String.format(\"Unable to abort task %s of job %s\", getTaskId(), getJobId()));\n      }\n    }\n\n    private RecordWriter<KeyT, ValueT> initRecordWriter(\n        OutputFormat<KeyT, ValueT> outputFormatObj, TaskAttemptContext taskAttemptContext)\n        throws IllegalStateException {\n      try {\n        LOG.info(\n            \"Creating new RecordWriter for task {} of Job with id {}.\",\n            taskAttemptContext.getTaskAttemptID().getTaskID().getId(),\n            taskAttemptContext.getJobID().getJtIdentifier());\n        return outputFormatObj.getRecordWriter(taskAttemptContext);\n      } catch (InterruptedException | IOException e) {\n        throw new IllegalStateException(\"Unable to create RecordWriter object: \", e);\n      }\n    }\n","sourceCodeStart":1539,"sourceCodeEnd":1575,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/hadoop-format/src/main/java/org/apache/beam/sdk/io/hadoop/format/HadoopFormatIO.java#L1539-L1575","documentation":"HadoopFormatIO's sink task wrapper calls abortTask() when a task fails, delegating to the OutputCommitter. If abortTask itself throws an IOException, the error is converted to this IllegalStateException — notably the original cause is not chained, so the message only identifies task and job ids.","triggerScenarios":"Failure cleanup path: the task failed, and OutputCommitter.abortTask(taskAttemptContext) threw an IOException (e.g., filesystem unavailability while deleting temporary output).","commonSituations":"HDFS/cluster unavailability during job cleanup; misconfigured OutputCommitter that cannot abort; temporary-output directories already removed or permission issues.","solutions":["Check the earlier IOException in the logs — this error masks the real abort failure cause","Verify the output filesystem is reachable and the committer's temp paths exist/are writable","Use a standard OutputCommitter (e.g., FileOutputCommitter) compatible with the Hadoop version"],"exampleFix":"// before\nthrow new IllegalStateException(String.format(\"Unable to abort task %s of job %s\", getTaskId(), getJobId()));\n// after\nthrow new IllegalStateException(String.format(\"Unable to abort task %s of job %s\", getTaskId(), getJobId()), e); // chain cause","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  writeResult();\n} catch (IllegalStateException e) {\n  if (e.getMessage().startsWith(\"Unable to abort task\")) {\n    LOG.error(\"Cleanup failed; check filesystem availability and output committer config\", e);\n    throw e;\n  }\n  throw e;\n}","preventionTips":["Verify output filesystem reachability before job submission","Use the standard FileOutputCommitter for your Hadoop version","Keep job cleanup from racing with other cleanup jobs (unique run ids)"],"tags":["hadoop","outputcommitter","abort","sink"],"backgroundTag":"file-write-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}