apache/hadoop · error · IOException

Storage type %s already exists on same mount: %s.

Error message

Storage type %s already exists on same mount: %s.

What it means

Thrown as IOException from FsDatasetImpl.addVolume when the same-disk-tiering mount check finds a volume already registered for the same mount point and StorageType. When multiple data dirs live on one mount, each (mount, storage type) pair may only be served by one volume; the check via volumes.getMountVolumeMap().getVolumeRefByMountAndStorageType(mount, storageType) returns an existing reference and this add is rejected. The reference is closed before throwing so no leak is left behind.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java:467

          "Found duplicated storage UUID: %s in %s.",
          sd.getStorageUuid(), sd.getVersionFile());
      LOG.error(errorMsg);
      throw new IOException(errorMsg);
    }
    // Check if there is same storage type on the mount.
    // Only useful when same disk tiering is turned on.
    FsVolumeImpl volumeImpl = (FsVolumeImpl) ref.getVolume();
    FsVolumeReference checkRef = volumes
        .getMountVolumeMap()
        .getVolumeRefByMountAndStorageType(
            volumeImpl.getMount(), volumeImpl.getStorageType());
    if (checkRef != null) {
      final String errorMsg = String.format(
          "Storage type %s already exists on same mount: %s.",
          volumeImpl.getStorageType(), volumeImpl.getMount());
      checkRef.close();
      LOG.error(errorMsg);
      throw new IOException(errorMsg);
    }
    volumeMap.mergeAll(replicaMap);
    storageMap.put(sd.getStorageUuid(),
        new DatanodeStorage(sd.getStorageUuid(),
            DatanodeStorage.State.NORMAL,
            storageType));
    asyncDiskService.addVolume(volumeImpl);
    volumes.addVolume(ref);
  }

  private void addVolume(Storage.StorageDirectory sd) throws IOException {
    final StorageLocation storageLocation = sd.getStorageLocation();

    // If IOException raises from FsVolumeImpl() or getVolumeMap(), there is
    // nothing needed to be rolled back to make various data structures, e.g.,
    // storageMap and asyncDiskService, consistent.
    FsVolumeImpl fsVolume = new FsVolumeImplBuilder()
                              .setDataset(this)

View on GitHub (pinned to 2add963021)

Solutions

1. Identify the duplicate: the message names the storage type and mount; list all dfs.datanode.data.dir entries on that mount and their [TYPE] tags.
2. Keep exactly one dir per (mount, storage type): either merge the duplicate dirs into one, or retag one as a different type ([SSD]/[ARCHIVE]/[RAM_DISK]) if tiering on one disk is the goal.
3. Put each dir on its own real mount (separate disks/filesystems) if you intended independent volumes.
4. If you did not intend same-disk tiering, verify dfs.datanode.same-disk-tiering.enabled and the mount detection (df output) for those dirs, then restart the DataNode.

Example fix

<!-- before: two DISK dirs on the same mount /mnt/big -->
<property>
  <name>dfs.datanode.data.dir</name>
  <value>[DISK]/mnt/big/dn1,[DISK]/mnt/big/dn2</value>
</property>

<!-- after: one dir per (mount, storage type) -->
<property>
  <name>dfs.datanode.data.dir</name>
  <value>[DISK]/mnt/big/dn1,[ARCHIVE]/mnt/big/dn2</value>
</property>

// Before enabling same-disk-tiering, assert (mount, storageType) uniqueness.
Map<String, String> mountToType = new HashMap<>();
for (String d : conf.getTrimmedStrings("dfs.datanode.data.dir")) {
  Matcher m = Pattern.compile("^\[([A-Z_]+)\](.*)$").matcher(d);
  String type = m.matches() ? m.group(1) : "DISK";
  File f = new File(m.matches() ? m.group(2) : d).getAbsoluteFile();
  String mount = Files.getFileStore(f.toPath()).name();
  String prev = mountToType.putIfAbsent(mount + ":" + type, d);
  if (prev != null) {
    throw new IOException(type + " declared twice on mount " + mount
        + ": " + prev + " and " + d);
  }
}

When it happens

Trigger: Configuring dfs.datanode.data.dir with two dirs on the same mount that declare the same storage type, e.g. [DISK]/mnt/big/dn1 and [DISK]/mnt/big/dn2 where /mnt/big is one filesystem, while dfs.datanode.same-disk-tiering.enabled is active. addVolume for the second dir finds the first in the mount-volume map and throws.

Common situations: Squeezing multiple 'volumes' out of one large disk without distinct storage types; migrating from separate disks to LVM/single XFS and forgetting to merge dirs; enabling same-disk-tiering on a config that predates it; specifying [DISK]/ and [DISK]/data on the root filesystem.

Related errors

• Failed to hardLink {srcReplica} block file to {dstFile}
• Changing storage type is not allowed.
• Disk mount {mount} already has volume, when trying to add {l
• Found duplicated storage UUID: %s in %s.
• BlockId {} is not valid.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/363f7cbdf75964f9. Report an issue: GitHub.