{"record":{"id":"ce9371fe9b73de7d","repo":"apache/hadoop","slug":"hadoop-bin-directory-does-not-exist","errorCode":null,"errorMessage":"Hadoop bin directory does not exist: {}","messagePattern":"Hadoop bin directory does not exist: (.+?)","errorType":"exception","errorClass":"FileNotFoundException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java","lineNumber":661,"sourceCode":"      throws FileNotFoundException {\n    // construct hadoop bin path to the specified executable\n    return getQualifiedBinInner(getHadoopHomeDir(), executable);\n  }\n\n  /**\n   * Inner logic of {@link #getQualifiedBin(String)}, accessible\n   * for tests.\n   * @param hadoopHomeDir home directory (assumed to be valid)\n   * @param executable executable\n   * @return path to the binary\n   * @throws FileNotFoundException if the executable was not found/valid\n   */\n  static File getQualifiedBinInner(File hadoopHomeDir, String executable)\n      throws FileNotFoundException {\n    String binDirText = \"Hadoop bin directory \";\n    File bin = new File(hadoopHomeDir, \"bin\");\n    if (!bin.exists()) {\n      throw new FileNotFoundException(addOsText(binDirText + E_DOES_NOT_EXIST\n          + \": \" + bin));\n    }\n    if (!bin.isDirectory()) {\n      throw new FileNotFoundException(addOsText(binDirText + E_NOT_DIRECTORY\n          + \": \" + bin));\n    }\n\n    File exeFile = new File(bin, executable);\n    if (!exeFile.exists()) {\n      throw new FileNotFoundException(\n          addOsText(E_NO_EXECUTABLE + \": \" + exeFile));\n    }\n    if (!exeFile.isFile()) {\n      throw new FileNotFoundException(\n          addOsText(E_NOT_EXECUTABLE_FILE + \": \" + exeFile));\n    }\n    try {\n      return exeFile.getCanonicalFile();","sourceCodeStart":643,"sourceCodeEnd":679,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/Shell.java#L643-L679","documentation":"Shell.getQualifiedBinInner() resolves <hadoopHome>/bin/<executable> for tools like winutils.exe. After the home directory passes validation, it checks that the bin subdirectory exists; if ${HADOOP_HOME}/bin is absent it throws FileNotFoundException with the OS-prefixed message. The Hadoop install is incomplete: home is fine, but the directory holding the scripts/binaries is missing.","triggerScenarios":"Calling Shell.getQualifiedBin() / getWinutilsPath() (directly or via Windows LocalFileSystem operations such as creating temp dirs or setting file permissions) when ${HADOOP_HOME}/bin does not exist; trimmed-down distributions that shipped only jars.","commonSituations":"Slim Docker images that copy hadoop jars but delete bin/ to save space; hand-assembled HADOOP_HOME folders containing only etc/ and share/; Windows setups where a winutils zip was extracted into the wrong level so bin/ landed elsewhere.","solutions":["Verify the expected location: ls \"$HADOOP_HOME/bin\" — it must be a readable directory","If missing, re-extract the full distribution or copy a version-matched bin/ directory into HADOOP_HOME","Check whether bin/ actually exists one level up/down and repoint HADOOP_HOME to the directory that directly contains bin/","On Windows, replace the ad-hoc folder with a proper winutils distribution layout (%HADOOP_HOME%\\bin\\winutils.exe)"],"exampleFix":"# before\nHADOOP_HOME=/opt/hadoop-min   # contains jars only, no bin/\n\n# after\ncp -r /opt/hadoop-3.3.6-full/bin /opt/hadoop-min/\nls /opt/hadoop-min/bin   # bin/ now present","handlingStrategy":"validation","validationCode":"File home = new File(System.getenv(\"HADOOP_HOME\"));\nFile bin = new File(home, \"bin\");\nif (!bin.isDirectory()) {\n  throw new IllegalStateException(\"Missing \" + bin + \" — incomplete Hadoop installation\");\n}","typeGuard":null,"tryCatchPattern":"try { Shell.getQualifiedBin(\"winutils.exe\"); } catch (FileNotFoundException e) { /* provision bin/ then retry once */ }","preventionTips":["Never trim bin/ out of HADOOP_HOME when slimming images — Shell needs it for tool lookup","Verify bin/ exists in the same startup check that validates HADOOP_HOME","For Windows, install a full winutils distribution so %HADOOP_HOME%\\bin is populated before the app starts"],"tags":["hadoop","shell","bin-directory","winutils","filenotfound"],"backgroundTag":"invalid-hadoop-home","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}