apache/hadoop · error · IOException

Disk mount {mount} already has volume, when trying to add {l

Error message

Disk mount {mount} already has volume, when trying to add {location}. Please try removing mounts first or restart datanode.

What it means

With dfs.datanode.allow.same.disk.tiering enabled, refreshVolumes resolves each new location's mount point with DF and rejects the add when data.getMountVolumeMap() already has a volume on that mount — one volume per mount is enforced for tierable storage types. The IOException is explicit about the remedy: remove the existing volume on that mount first, or restart the DataNode.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java:1271

        if (StorageType.allowSameDiskTiering(location.getStorageType())) {
          File dir = new File(location.getUri());
          // Get the first parent dir that exists to check disk mount point.
          while (!dir.exists()) {
            dir = dir.getParentFile();
            if (dir == null) {
              throw new IOException("Invalid path: "
                  + location + ": directory does not exist");
            }
          }
          DF df = new DF(dir, dnConf.getConf());
          String mount = df.getMount();
          if (data.getMountVolumeMap().hasMount(mount)) {
            String errMsg = "Disk mount " + mount
                + " already has volume, when trying to add "
                + location + ". Please try removing mounts first"
                + " or restart datanode.";
            LOG.error(errMsg);
            throw new IOException(errMsg);
          }
        }
      }
    }
  }

  /**
   * Attempts to reload data volumes with new configuration.
   * @param newVolumes a comma separated string that specifies the data volumes.
   * @throws IOException on error. If an IOException is thrown, some new volumes
   * may have been successfully added and removed.
   */
  private void refreshVolumes(String newVolumes) throws IOException {
    // Add volumes for each Namespace
    final List<NamespaceInfo> nsInfos = Lists.newArrayList();
    for (BPOfferService bpos : blockPoolManager.getAllNamenodeThreads()) {
      nsInfos.add(bpos.getNamespaceInfo());
    }

View on GitHub (pinned to 2add963021)

Solutions

Remove the existing volume on that mount first: reconfig with it omitted from dfs.datanode.data.dir, let it deactivate, then add the new location in a second reconfig
If two dirs on one physical disk is intentional, review whether dfs.datanode.allow.same.disk.tiering should be enabled at all for this layout
Verify which volume owns the mount: findmnt/lsblk on the host plus the DN's 'Volume' JMX beans

Example fix

# before: both dirs on mount /mnt/disk2 -> rejected
[DISK]file:///mnt/disk2/dn0,[DISK]file:///mnt/disk2/dn1
# after: one volume per mount
[DISK]file:///mnt/disk2/dn0,[DISK]file:///mnt/disk3/dn1

Defensive patterns

Strategy: validation

Validate before calling

Set<String> usedMounts = new HashSet<>();
for (StorageLocation l : currentLocations) usedMounts.add(new DF(new File(l.getUri()), conf).getMount());
for (StorageLocation l : newLocations) {
  String m = new DF(new File(l.getUri()), conf).getMount();
  if (usedMounts.contains(m)) throw new IllegalArgumentException("mount " + m + " already has a volume");
}

Try / catch

catch (ReconfigurationException e) {
  if (String.valueOf(e.getCause()).contains("already has volume")) {
    // drop the old volume on that mount first, then re-add in a second reconfig
  }
}

Prevention

One DN volume per disk mount when same-disk tiering is enabled
Map every candidate dir to its device (findmnt/lsblk) before adding volumes
Watch for bind mounts and symlinks resolving two paths to one device

When it happens

Trigger: Adding [DISK]file:///mnt/disk2/dn1 when /mnt/disk2 already backs an active volume such as /mnt/disk2/dn0; bind mounts or symlinked paths making two directories resolve to the same device; partitions of one disk exposed under two mount paths.

Common situations: Operators splitting one big disk into multiple DN dirs for scan parallelism; containers with shared hostPath devices; SAN LUNs presented under two mount points; adding a 'spare' dir on the root disk.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5a9f1b24d2a93b0b. Report an issue: GitHub.