{"record":{"id":"61968732f06c38a4","repo":"apache/druid","slug":"cannot-list-files-in-directory-s","errorCode":null,"errorMessage":"Cannot list files in directory[%s]","messagePattern":"Cannot list files in directory\\[(.+?)\\]","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/utils/CompressionUtils.java","lineNumber":315,"sourceCode":"   */\n  public static long lz4CompressDirectory(File directory, OutputStream out) throws IOException\n  {\n    if (!directory.isDirectory()) {\n      throw new IOE(\"directory[%s] is not a directory\", directory);\n    }\n\n    // Use fast compressor for better performance (lower CPU, faster compression)\n    final LZ4BlockOutputStream lz4Out = new LZ4BlockOutputStream(\n        out,\n        64 * 1024, // Block size\n        LZ4Factory.fastestInstance().fastCompressor()\n    );\n    // Use DataOutputStream for structured writing\n    final DataOutputStream dataOut = new DataOutputStream(lz4Out);\n    final File[] files = directory.listFiles();\n\n    if (files == null) {\n      throw new IOE(\"Cannot list files in directory[%s]\", directory);\n    }\n\n    // Sort for consistency\n    final File[] sortedFiles = Arrays.stream(files).sorted().toArray(File[]::new);\n\n    dataOut.writeInt(sortedFiles.length);\n\n    long totalSize = 0;\n\n    for (File file : sortedFiles) {\n      if (file.isDirectory()) {\n        continue; // Skip subdirectories like ZIP does\n      }\n\n      log.debug(\"Compressing file[%s] with size[%,d]. Total size so far[%,d]\", file, file.length(), totalSize);\n\n      final String fileName = file.getName();\n      final byte[] fileNameBytes = fileName.getBytes(StandardCharsets.UTF_8);","sourceCodeStart":297,"sourceCodeEnd":333,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/utils/CompressionUtils.java#L297-L333","documentation":"Thrown by CompressionUtils.lz4CompressDirectory when File.listFiles() returns null for the directory being LZ4-compressed. Java's listFiles() returns null when the File is not a directory or cannot be read (missing, permission denied, or an I/O error). Druid throws this instead of a cryptic NPE so the caller knows the directory could not be enumerated.","triggerScenarios":"Calling compressDirectory (LZ4 mode) with a directory path that does not exist, points to a regular file instead of a directory, or the process lacks read permission on the directory.","commonSituations":"Task working directories deleted underneath a running ingestion task (container cleanup), wrong path in loadSpec/durable storage config, running Druid as a user without permissions on the segment cache dir, or a race where the directory is removed between existence check and listFiles().","solutions":["Verify the directory exists and is readable by the Druid process user before calling compressDirectory (e.g. `File dir = new File(path); if (!dir.isDirectory()) throw ...;`).","Check filesystem permissions (ls -ld) and fix ownership/ACLs on the directory.","If the path should exist, check whether another process (task cleanup, tmpwatch) deleted it; re-create or restart the task.","Pass a File that actually points to a directory, not a file path."],"exampleFix":"// before\nCompressionUtils.lz4CompressDirectory(..., new File(\"/tmp/segments\"), ...);\n// after\nfinal File dir = new File(\"/tmp/segments\");\nif (!dir.isDirectory()) {\n  throw new IllegalStateException(\"Not a readable directory: \" + dir);\n}\nCompressionUtils.lz4CompressDirectory(..., dir, ...);","handlingStrategy":"validation","validationCode":"final File dir = new File(path);\nif (!dir.isDirectory()) {\n  throw new IllegalArgumentException(\"Not a readable directory: \" + dir);\n}\nif (!dir.canRead()) {\n  throw new IOException(\"No read permission on: \" + dir);\n}","typeGuard":"static boolean isListableDirectory(File f) {\n  return f != null && f.isDirectory() && f.canRead();\n}","tryCatchPattern":"try {\n  CompressionUtils.compressDirectory(...);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"Cannot list files in directory\")) {\n    // recreate/repair directory or fail task with clear message\n  }\n  throw e;\n}","preventionTips":["Verify the directory with File.isDirectory() before compressing","Run Druid under a user with read access to all data directories","Avoid having external cleanup jobs delete directories mid-task","Canonicalize and log the path to catch config typos early"],"tags":["io","filesystem","compression"],"backgroundTag":"directory-not-found","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}