apache/hadoop · warning · IOException

Storage directory is in use.

Error message

Storage directory is in use.

What it means

DataStorage.prepareVolume is the runtime hot-swap path for adding volumes. containsStorageDir(location) returns true when the location is already registered in this DataStorage, so re-adding it would corrupt volume bookkeeping; the call fails fast with 'Storage directory is in use.' before touching the disk.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java:330

   * to the volume. If the volume cannot be added, it is OK to discard the
   * builder later.
   *
   * @param datanode DataNode object.
   * @param location the StorageLocation for the storage directory.
   * @param nsInfos an array of namespace infos.
   * @return a VolumeBuilder that holds the metadata of this storage directory
   * and can be added to DataStorage later.
   * @throws IOException if encounters I/O errors.
   *
   * Note that if there is IOException, the state of DataStorage is not modified.
   */
  public VolumeBuilder prepareVolume(DataNode datanode,
      StorageLocation location, List<NamespaceInfo> nsInfos)
          throws IOException {
    if (containsStorageDir(location)) {
      final String errorMessage = "Storage directory is in use.";
      LOG.warn(errorMessage);
      throw new IOException(errorMessage);
    }

    StorageDirectory sd = loadStorageDirectory(
        datanode, nsInfos.get(0), location, StartupOption.HOTSWAP, null);
    VolumeBuilder builder =
        new VolumeBuilder(this, sd);
    for (NamespaceInfo nsInfo : nsInfos) {
      location.makeBlockPoolDir(nsInfo.getBlockPoolID(), datanode.getConf());

      final BlockPoolSliceStorage bpStorage = getBlockPoolSliceStorage(nsInfo);
      final List<StorageDirectory> dirs = bpStorage.loadBpStorageDirectories(
          nsInfo, location, StartupOption.HOTSWAP, null, datanode.getConf());
      builder.addBpStorageDirectories(nsInfo.getBlockPoolID(), dirs);
    }
    return builder;
  }

  static int getParallelVolumeLoadThreadsNum(int dataDirs, Configuration conf) {

View on GitHub (pinned to 2add963021)

Solutions

1. Diff the new dfs.datanode.data.dir against the DN's currently effective config and add only genuinely new paths
2. If the volume is already active, no action is needed - the error just confirms it is in use
3. Resolve canonical paths before comparing so the same device is not added twice

// Before dfsadmin -reconfig, diff current vs new data.dir
Set<String> current = getCurrentDataDirsFromDnJmx();   // Hadoop:service=DataNode,name=DataNodeInfo
Set<String> requested = parse(newPropertyValue);
for (String added : Sets.difference(requested, current)) {
  if (current.contains(Paths.get(added).toRealPath().toString())) {
    throw new IllegalStateException("Volume already in use: " + added);
  }
}

When it happens

Trigger: Reconfiguring dfs.datanode.data.dir at runtime (dfsadmin -reconfig datanode ...) with a location that is already part of the running volume set - duplicated entry, replayed add request, or a symlink resolving to an existing volume's path.

Common situations: Reconfig payloads that append to the old list instead of replacing it; retrying a volume-add that already succeeded; /mnt vs /media style symlink aliases pointing at the same device.

Related errors

• Storage directory with location {} does not exist
• Failed to remove %s: %s
• All specified directories have failed to load.
• file VERSION is invalid.
• has incompatible storage Id.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e611d3d75d1d246b. Report an issue: GitHub.