{"record":{"id":"a5e6030599f3908a","repo":"apache/hadoop","slug":"found-duplicated-storage-uuid-s-in-s","errorCode":null,"errorMessage":"Found duplicated storage UUID: %s in %s.","messagePattern":"Found duplicated storage UUID: (.+?) in (.+?)\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java","lineNumber":452,"sourceCode":"      ReplicaMap replicaMap,\n      Storage.StorageDirectory sd, StorageType storageType,\n      FsVolumeReference ref) throws IOException {\n    for (String bp : volumeMap.getBlockPoolList()) {\n      lockManager.addLock(LockLevel.VOLUME, bp, ref.getVolume().getStorageID());\n      List<String> allSubDirNameForDataSetLock = datasetSubLockStrategy.getAllSubLockNames();\n      for (String dir : allSubDirNameForDataSetLock) {\n        lockManager.addLock(LockLevel.DIR, bp, ref.getVolume().getStorageID(), dir);\n        LOG.info(\"Added DIR lock for bpid:{}, volume storageid:{}, dir:{}\",\n            bp, ref.getVolume().getStorageID(), dir);\n      }\n    }\n    DatanodeStorage dnStorage = storageMap.get(sd.getStorageUuid());\n    if (dnStorage != null) {\n      final String errorMsg = String.format(\n          \"Found duplicated storage UUID: %s in %s.\",\n          sd.getStorageUuid(), sd.getVersionFile());\n      LOG.error(errorMsg);\n      throw new IOException(errorMsg);\n    }\n    // Check if there is same storage type on the mount.\n    // Only useful when same disk tiering is turned on.\n    FsVolumeImpl volumeImpl = (FsVolumeImpl) ref.getVolume();\n    FsVolumeReference checkRef = volumes\n        .getMountVolumeMap()\n        .getVolumeRefByMountAndStorageType(\n            volumeImpl.getMount(), volumeImpl.getStorageType());\n    if (checkRef != null) {\n      final String errorMsg = String.format(\n          \"Storage type %s already exists on same mount: %s.\",\n          volumeImpl.getStorageType(), volumeImpl.getMount());\n      checkRef.close();\n      LOG.error(errorMsg);\n      throw new IOException(errorMsg);\n    }\n    volumeMap.mergeAll(replicaMap);\n    storageMap.put(sd.getStorageUuid(),","sourceCodeStart":434,"sourceCodeEnd":470,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java#L434-L470","documentation":"Thrown as IOException from FsDatasetImpl.addVolume during block-pool volume setup when a StorageDirectory's storage UUID is already present in storageMap. Each formatted storage dir gets a unique UUID in its VERSION file; two dirs presenting the same UUID means the DataNode is being pointed at the same storage twice. The message includes the offending UUID and the VERSION file path that carried the duplicate.","triggerScenarios":"Calling addVolume (DataNode startup / BPOfferService init) with a StorageDirectory whose getVersionFile() carries a storage UUID already registered for that block pool. Typical cause: the same physical directory (or a copy/clone of it) listed twice under dfs.datanode.data.dir, or a bind mount/symlink alias of a dir that is also listed directly.","commonSituations":"Same dir listed twice in dfs.datanode.data.dir (e.g. file:// and plain path forms); a disk cloned with dd/rsync so both copies share a VERSION UUID; symlinks like /data/dn -> /mnt/disk1 with both paths configured; leftover duplicate entries after config refactoring.","solutions":["Read the message: open the named VERSION file, note the storageID, then grep every dir's current/VERSION for the same storageID to find both occurrences.","Deduplicate dfs.datanode.data.dir: remove the alias/symlink/duplicate entry so each physical dir appears exactly once.","If a dir was cloned, reformat the redundant copy (hdfs datanode -format or delete its VERSION/subdirs) so it gets a fresh UUID.","Restart the DataNode and verify each configured dir maps to a distinct UUID."],"exampleFix":"<!-- before: same physical dir via two spellings -->\n<property>\n  <name>dfs.datanode.data.dir</name>\n  <value>/data/dn,[DISK]/data/dn,/mnt/disk1/dn</value>\n</property>\n\n<!-- after: one entry per physical directory -->\n<property>\n  <name>dfs.datanode.data.dir</name>\n  <value>/mnt/disk1/dn</value>\n</property>","handlingStrategy":"validation","validationCode":"// Before DN start, assert every configured dir has a distinct storage UUID.\nSet<String> seen = new HashSet<>();\nfor (String d : conf.getTrimmedStrings(\"dfs.datanode.data.dir\")) {\n  File v = new File(d.replaceAll(\"^\\[[A-Z_]+\\]\", \"\"), \"current/VERSION\");\n  if (!v.isFile()) continue; // unformatted dirs get fresh UUIDs\n  Properties p = new Properties();\n  try (FileInputStream in = new FileInputStream(v)) { p.load(in); }\n  String id = p.getProperty(\"storageID\");\n  if (id != null && !seen.add(id)) {\n    throw new IOException(\"Duplicate storageID \" + id + \" at \" + v\n        + \" - deduplicate dfs.datanode.data.dir\");\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never list the same physical dir twice in dfs.datanode.data.dir, including via symlink or bind mount aliases.","When cloning a disk image, delete the clone's VERSION/current so it re-formats with a new UUID.","Use canonical paths (no mixed symlink/direct spellings) in the config.","On startup failure, grep 'Found duplicated storage UUID' — it names the exact VERSION file to inspect."],"tags":["hadoop","hdfs","datanode","fsdataset","storage-uuid","duplicate-config"],"backgroundTag":"duplicate-storage-uuid","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}