apache/druid · error · IOException

Cannot list directory [%s]

Error message

Cannot list directory [%s]

What it means

LocalDataSegmentPusher.pushNoZip throws IOException when File.listFiles() returns null on the input segment directory, meaning the directory could not be read (does not exist, is not a directory, or is unreadable). The no-zip push mode needs to enumerate all segment files to link them into storage.

Source

Thrown at server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPusher.java:136

      }

      return baseSegment.withLoadSpec(makeLoadSpec(new File(outDir, INDEX_ZIP_FILENAME).toURI()))
                        .withSize(size);
    }
    finally {
      FileUtils.deleteDirectory(tmpSegmentDir);
    }
  }

  private DataSegment pushNoZip(final File inDir, final File outDir, final DataSegment baseSegment) throws IOException
  {
    final File tmpSegmentDir = new File(config.getStorageDirectory(), makeIntermediateDir());
    FileUtils.mkdirp(tmpSegmentDir);

    try {
      final File[] files = inDir.listFiles();
      if (files == null) {
        throw new IOE("Cannot list directory [%s]", inDir);
      }

      long size = 0;
      for (final File file : files) {
        if (file.isFile()) {
          size += file.length();
          FileUtils.linkOrCopy(file, new File(tmpSegmentDir, file.getName()));
        } else {
          // Segment directories are expected to be flat.
          throw new IOE("Unexpected subdirectory [%s]", file.getName());
        }
      }

      final File segmentDir = new File(outDir, INDEX_DIR);
      FileUtils.mkdirp(outDir);

      try {
        Files.move(tmpSegmentDir.toPath(), segmentDir.toPath(), StandardCopyOption.ATOMIC_MOVE);

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Verify inDir exists, is a directory, and is readable by the Druid user (ls -ld <dir>).
2. Fix filesystem permissions/ownership of the task work directory.
3. Check that the mount holding the segment dir is healthy (NFS/disk errors).
4. Re-run the ingestion task to regenerate the segment directory.

Example fix

// before: dir deleted before push (aggressive cleanup job)
find /druid-task-work -mmin +30 -delete
// after: retain until push completes / exclude task dirs
find /druid-task-work -mmin +1440 -delete

if (!inDir.isDirectory() || !inDir.canRead()) {
  throw new IllegalStateException("segment input dir missing or unreadable: " + inDir);
}

boolean isReadableDirectory(File dir) {
  return dir != null && dir.isDirectory() && dir.canRead() && dir.canExecute();
}

try {
  pusher.push(segment, outDir, false);
} catch (IOException e) {
  log.error(e, "cannot read segment dir %s; check permissions/mount", inDir);
  throw e;
}

When it happens

Trigger: pushToPath -> pushNoZip with an inDir that is missing, is actually a file, or lacks read/execute permission; also on I/O errors from the underlying filesystem.

Common situations: Task output directory cleaned up before push; permissions broken after running tasks under different users; path misconfigured so a file path is passed where a directory is expected; NFS mount dropped.

Related errors

• Failed to rename [%s] to [%s]
• Unable to load from local directory [%s]
• Unexpected subdirectory [%s]
• RuntimeException wrapping underlying cause
• Input stream is null

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/87725ca92fa6dd69. Report an issue: GitHub.