{"record":{"id":"aee283d50ee2c9a2","repo":"apache/hadoop","slug":"could-not-parse-history-file-historyfileabsolute","errorCode":null,"errorMessage":"Could not parse history file ${historyFileAbsolute}","messagePattern":"Could not parse history file (.+?)","errorType":"exception","errorClass":"YarnRuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java","lineNumber":391,"sourceCode":"    if (this.jobInfo != null) {\n      return;\n    }\n    \n    if (historyFileAbsolute != null) {\n      JobHistoryParser parser = null;\n      try {\n        parser = createJobHistoryParser(historyFileAbsolute);\n        this.jobInfo = parser.parse();\n      } catch (IOException e) {\n        String errorMsg = \"Could not load history file \" + historyFileAbsolute;\n        LOG.warn(errorMsg, e);\n        throw new YarnRuntimeException(errorMsg, e);\n      }\n      IOException parseException = parser.getParseException(); \n      if (parseException != null) {\n        String errorMsg = \"Could not parse history file \" + historyFileAbsolute;\n        LOG.warn(errorMsg, parseException);\n        throw new YarnRuntimeException(errorMsg, parseException);\n      }\n    } else {\n      String errorMsg = \"History file not found\";\n      LOG.warn(errorMsg);\n      throw new IOException(errorMsg);\n    }\n    if (loadTasks) {\n      loadAllTasks();\n      LOG.info(\"TaskInfo loaded\");\n    }    \n  }\n\n  @Override\n  public List<String> getDiagnostics() {\n    return Collections.singletonList(jobInfo.getErrorInfo());\n  }\n\n  @Override","sourceCodeStart":373,"sourceCodeEnd":409,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java#L373-L409","documentation":"JobHistoryParser.parse() returned but stored a deferred parse exception (parser.getParseException() != null), meaning the .jhist content is malformed — truncated event stream or invalid syntax. CompletedJob wraps it in YarnRuntimeException('Could not parse history file ...') and the job becomes unloadable. This differs from 4401: the file was readable, its content is simply not valid history.","triggerScenarios":"Loading a job whose .jhist is truncated (AM killed with SIGKILL before history flush finished, a JSON/Avro line cut mid-write), corrupted by disk or HDFS bit rot, or written by an incompatible Hadoop version whose event schema differs.","commonSituations":"Jobs killed -9 right before completion so history flush is interrupted; .jhist files copied between clusters running different Hadoop versions; failing datanode disks corrupting under-replicated history blocks; history files touched by scripts.","solutions":["Read the wrapped parse exception in the JHS log — it shows the offending position/event in the file","Confirm truncation manually: hdfs dfs -cat <file>.jhist | tail — an unterminated last line confirms a partial write","Delete or quarantine the corrupt .jhist (it will never parse) so the JHS stops retrying it","If history migrates between clusters, keep both on compatible Hadoop versions or convert the files"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// cheap sanity check before parse: non-empty and last byte terminates the record stream\nFileStatus st = fc.getFileStatus(historyFileAbsolute);\nif (st.getLen() == 0) {\n  // zero-length .jhist — will never parse, quarantine early\n}","typeGuard":null,"tryCatchPattern":"try {\n  job = history.getJob(jobId);\n} catch (YarnRuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Could not parse history file\")) {\n    quarantineHistoryFile(jobId); // move aside so the JHS stops failing on every access\n    return unavailableJob(jobId);\n  }\n  throw e;\n}","preventionTips":["Kill jobs via `mapred job -kill`, not kill -9 on the AM, so history flush completes","Do not copy .jhist files between clusters on incompatible Hadoop versions","Monitor datanode disk health and HDFS corrupt-block counts; bit rot shows up here first","Quarantine (move) rather than repeatedly retry unparseable .jhist files"],"tags":["mapreduce","job-history-server","jhist","corrupt-file","parser"],"backgroundTag":"corrupt-file-parse-error","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}