apache/druid · error · IOException (IOE)

Cannot list directory [%s]

Error message

Cannot list directory [%s]

What it means

S3DataSegmentPusher.pushNoZip lists the segment directory's files with File.listFiles(); a null return means the directory could not be read (does not exist, is a file, or I/O error). The pusher throws this IOException instead of silently pushing nothing.

Source

Thrown at extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentPusher.java:114

      }
      catch (S3Exception e) {
        throw handlePushServiceException(e, indexSize);
      }
      catch (Exception e) {
        throw new RuntimeException(e);
      }
    }
    finally {
      log.debug("Deleting temporary cached index.zip");
      zipOutFile.delete();
    }
  }

  private DataSegment pushNoZip(File indexFilesDir, DataSegment baseSegment, String s3Path) throws IOException
  {
    final File[] files = indexFilesDir.listFiles();
    if (files == null) {
      throw new IOE("Cannot list directory [%s]", indexFilesDir);
    }

    long size = 0;
    for (final File file : files) {
      if (file.isFile()) {
        size += file.length();

        try {
          S3Utils.retryS3Operation(
              () -> {
                S3Utils.uploadFileIfPossible(s3Client, config.getDisableAcl(), config.getBucket(), s3Path + file.getName(), file);
                return null;
              }
          );
        }
        catch (S3Exception e) {
          throw handlePushServiceException(e, file.length());
        }

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Ensure the segment files directory exists and is readable before pushing.
2. Check filesystem permissions on the directory and its parents.
3. Look for concurrent cleanup jobs deleting the task's working directory.
4. Verify disk health (I/O errors also make listFiles return null).

Example fix

// before
pusher.push(new File("/missing/dir"), segment);
// after
File dir = new File("/missing/dir");
if (!dir.isDirectory()) {
  throw new IllegalStateException("Segment dir missing: " + dir);
}
pusher.push(dir, segment);

if (dir == null || !dir.isDirectory()) throw new IllegalArgumentException("not a readable directory: " + dir);

try { pusher.push(dir, segment); } catch (IOException e) { /* check directory readability before retrying */ }

When it happens

Trigger: Calling push/pushToPath with an indexFilesDir that was deleted, never created, or is not a directory, so File.listFiles() returns null.

Common situations: Task working directory cleaned up mid-push by external cleanup; permissions problem on the deep storage staging dir; bug in a custom segment publisher writing to the wrong path.

Related errors

• Could not fetch last modified timestamp from URI [%s]
• Unexpected subdirectory [%s]
• Failed to stream logs for task[%s] starting at offset[%d]
• Could not close channel for level [%d] and rank [%d]
• Unable to close channel for name :

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7ed44949a17764c2. Report an issue: GitHub.