apache/hadoop · error · IOException

Cannot list contents of {}

Error message

Cannot list contents of {}

What it means

IOException from DatanodeUtil.dirNoFilesRecursive, which walks a directory tree to decide whether it contains any regular files (used during DataNode startup/volume cleanup to identify empty directory structures that can be removed). Java's File.listFiles() returns null instead of an empty array when listing fails - the directory is unreadable or the path is not a directory - and that null is converted into this error.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DatanodeUtil.java:106

  }

  /** @return the unlink file. */
  public static File getUnlinkTmpFile(File f) {
    return new File(f.getParentFile(), f.getName()+UNLINK_BLOCK_SUFFIX);
  }

  /**
   * Checks whether there are any files anywhere in the directory tree rooted
   * at dir (directories don't count as files). dir must exist
   * @return true if there are no files
   * @throws IOException if unable to list subdirectories
   */
  public static boolean dirNoFilesRecursive(
      FsVolumeSpi volume, File dir,
      FileIoProvider fileIoProvider) throws IOException {
    File[] contents = fileIoProvider.listFiles(volume, dir);
    if (contents == null) {
      throw new IOException("Cannot list contents of " + dir);
    }
    for (File f : contents) {
      if (!f.isDirectory() ||
          (f.isDirectory() && !dirNoFilesRecursive(
              volume, f, fileIoProvider))) {
        return false;
      }
    }
    return true;
  }

  /**
   * Take an example.
   * We hava a block with blockid mapping to:
   * "/data1/hadoop/hdfs/datanode/current/BP-xxxx/current/finalized/subdir0/subdir1"
   * We return "subdir0/subdir0".
   * @param blockId the block id.
   * @return two-level subdir string where block will be stored.

View on GitHub (pinned to 2add963021)

Solutions

1. Check the exact directory named in the message: ls -ld <dir> as the datanode user to see EACCES vs ENOENT vs EIO
2. Restore read+execute permission/ownership on the subtree for the datanode user (namei -l <dir> shows which component fails)
3. If the directory was deleted by an external process, restart the DataNode so it rescans from a consistent snapshot
4. If EIO, treat as disk failure: check dmesg/SMART and follow volume-failure handling (evacuate and replace)

Example fix

# before: 'Cannot list contents of /data/dfs/current/BP-.../subdir7'
namei -l /data/dfs/current/BP-*/subdir7   # find the component with wrong perms

# after: fix the failing component and restart DN scan
sudo chown hdfs:hadoop /data/dfs/current/BP-XXXX/subdir7
sudo chmod u+rx /data/dfs/current/BP-XXXX/subdir7
hdfs --daemon restart datanode

// Before scans/cleanup, verify every path component is traversable
private static boolean traversable(File dir) {
  for (File f = dir; f != null; f = f.getParentFile()) {
    if (f.exists() && (!f.isDirectory() || !f.canRead() || !f.canExecute())) return false;
  }
  return true;
}
if (!traversable(targetDir)) throw new IOException("Directory tree not listable: " + targetDir);

try {
  DatanodeUtil.dirNoFilesRecursive(volume, dir, fileIoProvider);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Cannot list contents of")) {
    String d = e.getMessage().substring("Cannot list contents of ".length()).trim();
    // namei -l <d>: fix the failing component's perms, or handle ENOENT/EIO (disk)
  }
}

When it happens

Trigger: fileIoProvider.listFiles(volume, dir) returns null during recursive scanning: directory removed/renamed by another process mid-scan, permission loss on a subtree (EACCES), or I/O errors on a failing disk returning errors from readdir().

Common situations: DataNode startup scanning while a volume is half-broken (permissions changed by backup agents, NFS squashed roots), concurrent admin scripts pruning storage dirs, failing disks producing readdir errors.

Related errors

• Possible disk error: Failed to create {}
• Cannot remove directory {}
• Cannot remove directory {}
• Cannot create directory {}
• Mkdirs failed to create {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/075c4effe6173c68. Report an issue: GitHub.