{"record":{"id":"075c4effe6173c68","repo":"apache/hadoop","slug":"cannot-list-contents-of","errorCode":null,"errorMessage":"Cannot list contents of {}","messagePattern":"Cannot list contents of (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DatanodeUtil.java","lineNumber":106,"sourceCode":"  }\n\n  /** @return the unlink file. */\n  public static File getUnlinkTmpFile(File f) {\n    return new File(f.getParentFile(), f.getName()+UNLINK_BLOCK_SUFFIX);\n  }\n\n  /**\n   * Checks whether there are any files anywhere in the directory tree rooted\n   * at dir (directories don't count as files). dir must exist\n   * @return true if there are no files\n   * @throws IOException if unable to list subdirectories\n   */\n  public static boolean dirNoFilesRecursive(\n      FsVolumeSpi volume, File dir,\n      FileIoProvider fileIoProvider) throws IOException {\n    File[] contents = fileIoProvider.listFiles(volume, dir);\n    if (contents == null) {\n      throw new IOException(\"Cannot list contents of \" + dir);\n    }\n    for (File f : contents) {\n      if (!f.isDirectory() ||\n          (f.isDirectory() && !dirNoFilesRecursive(\n              volume, f, fileIoProvider))) {\n        return false;\n      }\n    }\n    return true;\n  }\n\n  /**\n   * Take an example.\n   * We hava a block with blockid mapping to:\n   * \"/data1/hadoop/hdfs/datanode/current/BP-xxxx/current/finalized/subdir0/subdir1\"\n   * We return \"subdir0/subdir0\".\n   * @param blockId the block id.\n   * @return two-level subdir string where block will be stored.","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DatanodeUtil.java#L88-L124","documentation":"IOException from DatanodeUtil.dirNoFilesRecursive, which walks a directory tree to decide whether it contains any regular files (used during DataNode startup/volume cleanup to identify empty directory structures that can be removed). Java's File.listFiles() returns null instead of an empty array when listing fails - the directory is unreadable or the path is not a directory - and that null is converted into this error.","triggerScenarios":"fileIoProvider.listFiles(volume, dir) returns null during recursive scanning: directory removed/renamed by another process mid-scan, permission loss on a subtree (EACCES), or I/O errors on a failing disk returning errors from readdir().","commonSituations":"DataNode startup scanning while a volume is half-broken (permissions changed by backup agents, NFS squashed roots), concurrent admin scripts pruning storage dirs, failing disks producing readdir errors.","solutions":["Check the exact directory named in the message: ls -ld <dir> as the datanode user to see EACCES vs ENOENT vs EIO","Restore read+execute permission/ownership on the subtree for the datanode user (namei -l <dir> shows which component fails)","If the directory was deleted by an external process, restart the DataNode so it rescans from a consistent snapshot","If EIO, treat as disk failure: check dmesg/SMART and follow volume-failure handling (evacuate and replace)"],"exampleFix":"# before: 'Cannot list contents of /data/dfs/current/BP-.../subdir7'\nnamei -l /data/dfs/current/BP-*/subdir7   # find the component with wrong perms\n\n# after: fix the failing component and restart DN scan\nsudo chown hdfs:hadoop /data/dfs/current/BP-XXXX/subdir7\nsudo chmod u+rx /data/dfs/current/BP-XXXX/subdir7\nhdfs --daemon restart datanode","handlingStrategy":"validation","validationCode":"// Before scans/cleanup, verify every path component is traversable\nprivate static boolean traversable(File dir) {\n  for (File f = dir; f != null; f = f.getParentFile()) {\n    if (f.exists() && (!f.isDirectory() || !f.canRead() || !f.canExecute())) return false;\n  }\n  return true;\n}\nif (!traversable(targetDir)) throw new IOException(\"Directory tree not listable: \" + targetDir);","typeGuard":null,"tryCatchPattern":"try {\n  DatanodeUtil.dirNoFilesRecursive(volume, dir, fileIoProvider);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Cannot list contents of\")) {\n    String d = e.getMessage().substring(\"Cannot list contents of \".length()).trim();\n    // namei -l <d>: fix the failing component's perms, or handle ENOENT/EIO (disk)\n  }\n}","preventionTips":["Freeze external scripts that prune/rename storage directories while the DataNode runs","Keep uniform datanode ownership on all storage dirs; audit after backup-agent or config-management runs","Treat repeated listFiles-null on one volume as an early disk-failure signal (check dmesg/SMART)"],"tags":["hdfs","datanode","directory-scan","listfiles-null","permissions","volume-health"],"backgroundTag":"directory-unreadable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}