{"record":{"id":"e6d79fe50820f570","repo":"apache/hadoop","slug":"invalid-path-location-directory-does-not-exist","errorCode":null,"errorMessage":"Invalid path: {location}: directory does not exist","messagePattern":"Invalid path: (.+?): directory does not exist","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java","lineNumber":1259,"sourceCode":"   *\n   * TODO: We can add feature to\n   *   allow refreshing volume with capacity ratio,\n   *   and solve the case of replacing volume on same mount.\n   */\n  private void validateVolumesWithSameDiskTiering(ChangedVolumes\n      changedVolumes) throws IOException {\n    if (dnConf.getConf().getBoolean(DFS_DATANODE_ALLOW_SAME_DISK_TIERING,\n        DFS_DATANODE_ALLOW_SAME_DISK_TIERING_DEFAULT)\n        && data.getMountVolumeMap() != null) {\n      // Check if mount already exist.\n      for (StorageLocation location : changedVolumes.newLocations) {\n        if (StorageType.allowSameDiskTiering(location.getStorageType())) {\n          File dir = new File(location.getUri());\n          // Get the first parent dir that exists to check disk mount point.\n          while (!dir.exists()) {\n            dir = dir.getParentFile();\n            if (dir == null) {\n              throw new IOException(\"Invalid path: \"\n                  + location + \": directory does not exist\");\n            }\n          }\n          DF df = new DF(dir, dnConf.getConf());\n          String mount = df.getMount();\n          if (data.getMountVolumeMap().hasMount(mount)) {\n            String errMsg = \"Disk mount \" + mount\n                + \" already has volume, when trying to add \"\n                + location + \". Please try removing mounts first\"\n                + \" or restart datanode.\";\n            LOG.error(errMsg);\n            throw new IOException(errMsg);\n          }\n        }\n      }\n    }\n  }\n","sourceCodeStart":1241,"sourceCodeEnd":1277,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java#L1241-L1277","documentation":"checkSameDiskTieringMount runs during refreshVolumes when dfs.datanode.allow.same.disk.tiering is true and the new location's storage type participates in same-disk tiering. To find the disk mount point it climbs parent-by-parent from the location path to the first existing directory; if getParentFile() returns null before any ancestor exists (relative path, a URI java.io.File cannot walk, or an entirely missing directory chain), it throws IOException(\"Invalid path: <location>: directory does not exist\").","triggerScenarios":"Submitting a relative data dir like data/dn1; a path under a mount that is not mounted or not yet created (typo /mnt/diskX/dn where diskX does not exist); a URI form File maps to nothing. Only checked when same-disk tiering is enabled and the location's type allows it.","commonSituations":"New disks cabled but not mounted when the reconfig fires; template paths referencing host-specific mounts that differ per host; NFS/network mounts flapping mid-refresh.","solutions":["Create the directory chain on the DN host first: mkdir -p /mnt/disk2/dn && chown hdfs:hdfs, then resubmit","Use absolute paths with the file:// scheme; relative paths cannot be resolved by this check","Verify the mount is live before retrying: findmnt /mnt/disk2 or stat -f /mnt/disk2"],"exampleFix":"# before: /mnt/diskX not mounted, path has no existing ancestor\ndfs.datanode.data.dir=[DISK]/mnt/diskX/dn\n# after\nmkdir -p /mnt/diskX/dn && mount /dev/sdb1 /mnt/diskX   # on the DN host\ndfs.datanode.data.dir=[DISK]file:///mnt/diskX/dn","handlingStrategy":"validation","validationCode":"File f = new File(location.getUri());\nif (!f.isAbsolute()) throw new IllegalArgumentException(\"use an absolute file:// path: \" + location);\nFile probe = f;\nwhile (probe != null && !probe.exists()) probe = probe.getParentFile();\nif (probe == null) throw new IllegalArgumentException(\"no existing ancestor for \" + f);","typeGuard":null,"tryCatchPattern":"catch (ReconfigurationException e) {\n  if (String.valueOf(e.getCause()).contains(\"directory does not exist\")) {\n    // mkdir -p the path (and mount the disk) on the DN host, then resubmit\n  }\n}","preventionTips":["Pre-flight 'mkdir -p' every new data dir on the DN host before a volume refresh","Always use absolute paths with an explicit file:// scheme","Verify mounts are live (findmnt) before adding volumes under them"],"tags":["hadoop","hdfs","datanode","volume-management","filesystem","path-validation"],"backgroundTag":"path-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}