{"record":{"id":"11252d655950364d","repo":"apache/hadoop","slug":"cannot-lock-storage-root-the-directory-is-alrea","errorCode":null,"errorMessage":"Cannot lock storage {root}. The directory is already locked","messagePattern":"Cannot lock storage (.+?)\\. The directory is already locked","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java","lineNumber":915,"sourceCode":"     * <p> Locking is not supported by all file systems.\n     * E.g., NFS does not consistently support exclusive locks.\n     * \n     * <p> If locking is supported we guarantee exclusive access to the\n     * storage directory. Otherwise, no guarantee is given.\n     * \n     * @throws IOException if locking fails\n     */\n    public void lock() throws IOException {\n      if (isShared()) {\n        LOG.info(\"Locking is disabled for {}\", this.root);\n        return;\n      }\n      FileLock newLock = tryLock();\n      if (newLock == null) {\n        String msg = \"Cannot lock storage \" + this.root\n          + \". The directory is already locked\";\n        LOG.info(msg);\n        throw new IOException(msg);\n      }\n      // Don't overwrite lock until success - this way if we accidentally\n      // call lock twice, the internal state won't be cleared by the second\n      // (failed) lock attempt\n      lock = newLock;\n    }\n\n    /**\n     * Attempts to acquire an exclusive lock on the storage.\n     * \n     * @return A lock object representing the newly-acquired lock or\n     * <code>null</code> if storage is already locked.\n     * @throws IOException if locking fails.\n     */\n    @SuppressWarnings(\"resource\")\n    FileLock tryLock() throws IOException {\n      boolean deletionHookAdded = false;\n      File lockF = new File(root, STORAGE_FILE_LOCK);","sourceCodeStart":897,"sourceCodeEnd":933,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/Storage.java#L897-L933","documentation":"StorageDirectory.lock() gives each HDFS daemon process an exclusive OS-level file lock on <storage-dir>/in_use.lock (FileChannel.tryLock) so two servers never share one storage directory. When tryLock() returns null the lock is already held, and lock() throws this IOException at startup. Note: if isShared() is true the lock is intentionally skipped and this error cannot occur.","triggerScenarios":"A second NameNode or DataNode starts with dfs.namenode.name.dir / dfs.datanode.data.dir pointing at a directory whose in_use.lock is held; the previous JVM is still running or was killed without releasing; an HA failover begins before the old process exits; a stale lock is left behind on NFS (where locks are advisory and can outlive the owner).","commonSituations":"Two daemons configured with overlapping storage directories; accidental double start (supervisor script, docker container restart reusing a mounted volume); NFS-mounted storage directories; a checkpoint/2NN process still holding the directory.","solutions":["Find the holder of <dir>/in_use.lock (lsof +D <storage-dir> or fuser <dir>/in_use.lock) and shut that process down cleanly, then restart","Verify only one NameNode/DataNode instance is configured to use that storage directory (check hdfs-site.xml name.dir/data.dir lists for overlap)","If no live process holds it (typical on NFS), restart the NFS lock manager or move storage to a local filesystem","For HA, confirm the active node fully exited before restarting the standby"],"exampleFix":"<!-- before: two daodes share a directory -->\n<property><name>dfs.namenode.name.dir</name><value>/mnt/shared/nn</value></property>\n<!-- after: each daemon gets its own storage directory -->\n<property><name>dfs.namenode.name.dir</name><value>/data/nn</value></property>","handlingStrategy":"try-catch","validationCode":"static boolean isStorageDirLocked(File dir) throws IOException {\n  File lockF = new File(dir, \"in_use.lock\");\n  try (RandomAccessFile raf = new RandomAccessFile(lockF, \"rw\");\n       FileChannel ch = raf.getChannel()) {\n    FileLock l = ch.tryLock();\n    if (l == null) return true;\n    l.release();\n    return false;\n  }\n}\n// run before daemon startup: if (isStorageDirLocked(dir)) abort with holder info","typeGuard":null,"tryCatchPattern":"try {\n  sd.lock();\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"already locked\")) {\n    // surface which process holds <dir>/in_use.lock (lsof) and abort startup cleanly\n  }\n  throw e;\n}","preventionTips":["Never list the same directory in two daemons' storage-dir settings","Always stop daemons with the provided scripts so the lock is released","Keep storage directories on local filesystems, not NFS, to avoid stale locks","In HA setups, wait for the old active to exit before starting a replacement"],"tags":["hdfs","storage-directory","file-lock","startup","namenode","datanode"],"backgroundTag":"file-lock-contention","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}