{"record":{"id":"b67bc07c23f4f321","repo":"apache/hadoop","slug":"failed-to-initialize-existing-directories","errorCode":null,"errorMessage":"Failed to initialize existing directories","messagePattern":"Failed to initialize existing directories","errorType":"exception","errorClass":"YarnRuntimeException","httpStatus":null,"severity":"critical","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java","lineNumber":101,"sourceCode":"  \n  @Override\n  protected void serviceInit(Configuration conf) throws Exception {\n    LOG.info(\"JobHistory Init\");\n    this.conf = conf;\n    this.appID = ApplicationId.newInstance(0, 0);\n    this.appAttemptID = RecordFactoryProvider.getRecordFactory(conf)\n        .newRecordInstance(ApplicationAttemptId.class);\n\n    moveThreadInterval = conf.getLong(\n        JHAdminConfig.MR_HISTORY_MOVE_INTERVAL_MS,\n        JHAdminConfig.DEFAULT_MR_HISTORY_MOVE_INTERVAL_MS);\n\n    hsManager = createHistoryFileManager();\n    hsManager.init(conf);\n    try {\n      hsManager.initExisting();\n    } catch (IOException e) {\n      throw new YarnRuntimeException(\"Failed to initialize existing directories\", e);\n    }\n\n    storage = createHistoryStorage();\n    \n    if (storage instanceof Service) {\n      ((Service) storage).init(conf);\n    }\n    storage.setHistoryFileManager(hsManager);\n\n    super.serviceInit(conf);\n  }\n\n  protected HistoryStorage createHistoryStorage() {\n    return ReflectionUtils.newInstance(conf.getClass(\n        JHAdminConfig.MR_HISTORY_STORAGE, CachedHistoryStorage.class,\n        HistoryStorage.class), conf);\n  }\n  ","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java#L83-L119","documentation":"During serviceInit, JobHistory constructs a HistoryFileManager and calls initExisting() to scan the intermediate-done and done directories for existing history files. Any IOException from that scan is wrapped in YarnRuntimeException 'Failed to initialize existing directories', which aborts JHS startup with the original cause attached. Typical roots are missing or unwritable history directories or HDFS unavailability.","triggerScenarios":"mapreduce.jobhistory.done-dir or mapreduce.jobhistory.intermediate-done-dir missing or unwritable for the JHS user; HDFS unreachable at startup; a file occupying a directory path; ACL denying traversal of the history tree.","commonSituations":"Fresh installs where history directories were never created; permission drift after HDFS changes; JHS starting while HDFS is down; done-dir relocated without creating the new location.","solutions":["Read the wrapped cause in the stack trace — it names the real failure.","Create the configured directories with correct ownership: hadoop fs -mkdir -p for both done and intermediate-done dirs, then chown/chmod for the JHS user.","Ensure the JHS user has rwx on both directory trees.","Start JHS only after HDFS is reachable and healthy."],"exampleFix":"# before: done dir missing or owned by another user\nhadoop fs -ls /mr-history/done\n\n# after: provision both history dirs for the mapred user\nhadoop fs -mkdir -p /mr-history/done /mr-history/done_intermediate\nhadoop fs -chown -R mapred:hadoop /mr-history","handlingStrategy":"try-catch","validationCode":"# pre-start readiness: history dirs exist and are writable\nfor d in /mr-history/done /mr-history/done_intermediate; do\n  hadoop fs -test -d $d || hadoop fs -mkdir -p $d\n  hadoop fs -test -w $d || echo \"$d not writable\"\ndone","typeGuard":null,"tryCatchPattern":"catch YarnRuntimeException during JHS init and inspect getCause():\ntry {\n  jobHistory.init(conf);\n} catch (org.apache.hadoop.yarn.exceptions.YarnRuntimeException e) {\n  Throwable c = e.getCause(); // original IOException names the failing directory\n  throw c instanceof IOException ? (IOException) c : e;\n}","preventionTips":["Provision done and intermediate-done directories in cluster setup.","Start JHS as a service dependent on HDFS availability.","Keep the JHS user's ownership of the history tree."],"tags":["mapreduce","job-history-server","startup","hdfs","permissions"],"backgroundTag":"service-startup-failure","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}