apache/hadoop · error · DiskErrorException

All directories in dfs.datanode.data.dir are invalid: {}

Error message

All directories in dfs.datanode.data.dir are invalid: {}

What it means

Error "All directories in dfs.datanode.data.dir are invalid: {}" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/StorageLocationChecker.java:239

      if (dataDirs.size() == failedLocations.size()) {
        throw new DiskErrorException("Too many failed volumes - "
            + "current valid volumes: " + goodLocations.size()
            + ", volumes configured: " + dataDirs.size()
            + ", volumes failed: " + failedLocations.size()
            + ", volume failures tolerated: " + maxVolumeFailuresTolerated);
      }
    } else {
      if (failedLocations.size() > maxVolumeFailuresTolerated) {
        throw new DiskErrorException("Too many failed volumes - "
            + "current valid volumes: " + goodLocations.size()
            + ", volumes configured: " + dataDirs.size()
            + ", volumes failed: " + failedLocations.size()
            + ", volume failures tolerated: " + maxVolumeFailuresTolerated);
      }
    }

    if (goodLocations.size() == 0) {
      throw new DiskErrorException("All directories in "
          + DFS_DATANODE_DATA_DIR_KEY + " are invalid: "
          + failedLocations);
    }

    return new ArrayList<>(goodLocations.keySet());
  }

  public void shutdownAndWait(int gracePeriod, TimeUnit timeUnit) {
    try {
      delegateChecker.shutdownAndWait(gracePeriod, timeUnit);
    } catch (InterruptedException e) {
      LOG.warn("StorageLocationChecker interrupted during shutdown.");
      Thread.currentThread().interrupt();
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Verify every directory listed in dfs.datanode.data.dir exists, is mounted, and is writable by the DataNode user.
2. Check dmesg and smartctl for disk failures; replace failed disks and restore the mount points.
3. Correct wrong paths or bad [storage-type] tags in dfs.datanode.data.dir in hdfs-site.xml, then restart the DataNode.

When it happens

Trigger: DataNode startup when every directory listed in dfs.datanode.data.dir is unreadable or failed. Avoid by ensuring at least one configured data directory is on a healthy, writable disk.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3beeabe083aceae9. Report an issue: GitHub.