What causes "No files found at [%s]" in apache/druid?

Verify the path exists on HDFS: `hdfs dfs -ls `; if absent, the segment is orphaned in the metadata store.

apache/druid · error · SegmentLoadingException

No files found at [%s]

Error message

No files found at [%s]

What it means

Inside the directory branch of getSegmentFiles, the retry lambda first checks fs.exists(path) and throws SegmentLoadingException("No files found at [%s]") when the HDFS directory has disappeared. This runs within RetryUtils.retry, but SegmentLoadingException is not matched by the retry predicate (which only retries IOException/HdfsIOException), so the failure surfaces immediately. It means the segment directory expected in deep storage no longer exists.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java:217

  FileUtils.FileCopyResult getSegmentFiles(final Path path, final File outDir) throws SegmentLoadingException
  {
    try {
      FileUtils.mkdirp(outDir);
    }
    catch (IOException e) {
      throw new SegmentLoadingException(e, "");
    }
    try {
      final FileSystem fs = path.getFileSystem(config);
      if (fs.getFileStatus(path).isDirectory()) {

        // --------    directory     ---------
        try {
          return RetryUtils.retry(
              () -> {
                if (!fs.exists(path)) {
                  throw new SegmentLoadingException("No files found at [%s]", path.toString());
                }

                final RemoteIterator<LocatedFileStatus> children = fs.listFiles(path, false);
                final FileUtils.FileCopyResult result = new FileUtils.FileCopyResult();
                while (children.hasNext()) {
                  final LocatedFileStatus child = children.next();
                  final Path childPath = child.getPath();
                  final String fname = childPath.getName();
                  if (fs.getFileStatus(childPath).isDirectory()) {
                    log.warn("[%s] is a child directory, skipping", childPath.toString());
                  } else {
                    final File outFile = new File(outDir, fname);
                    try (final FSDataInputStream in = fs.open(childPath)) {
                      NativeIO.chunkedCopy(in, outFile);
                    }
                    result.addFile(outFile);
                  }
                }

View on GitHub (pinned to 9b90983fd2)

Solutions

Verify the path exists on HDFS: `hdfs dfs -ls <path from loadSpec>`; if absent, the segment is orphaned in the metadata store.
Run the metadata-store/deep-storage consistency cleanup (kill tasks or the Druid 'unused segments' cleanup) so orphaned segment rows are removed.
Check whether a concurrent kill/retention job deleted the segment and re-ingest the affected intervals if the data is still needed.
Confirm druid.storage.storageDirectory matches where segments were actually pushed (a moved deep storage makes all loadSpec paths stale).
If the deletion was accidental, restore from HDFS snapshots or backups and re-trigger segment loading.

Example fix

// before: loadSpec points to /druid/segments/ds/2020_01_01... that was manually removed
// $ hdfs dfs -ls hdfs://nn/druid/segments/ds/...  -> No such file or directory
// after: remove orphan metadata rows so the segment is no longer served
// curl -X DELETE 'http://coordinator:8081/druid/coordinator/v1/datasources/ds?kill=true&interval=2020-01-01/2020-01-02'
// (or re-run ingestion to repopulate deep storage)

Defensive patterns

Strategy: validation

Validate before calling

Path p = new Path(loadSpecPath);
FileSystem fs = p.getFileSystem(config);
if (!fs.exists(p)) {
  markSegmentOrphan(segmentId); // remove from metadata store or re-ingest before loading
}

Try / catch

try { puller.getSegmentFiles(path, outDir); }
catch (SegmentLoadingException e) {
  if (e.getMessage().startsWith("No files found")) { dropOrphanSegment(segment); }
}

Prevention

Run Druid's built-in metadata/deep-storage consistency cleanup regularly.
Never manually delete segment directories without also killing metadata rows.
Keep druid.storage.storageDirectory stable; migrating deep storage requires updating loadSpec paths.
Order operations so retention/kill tasks complete before nodes attempt to load affected segments.

When it happens

Trigger: Loading a segment whose hdfs:// directory was concurrently deleted (kill task, killAll, retention policy, manual hdfs dfs -rm), or whose loadSpec path is wrong (typo in storageDirectory, moved/renamed deep storage location, cluster migration without copying segments).

Common situations: Historical trying to load a segment listed in metadata store but already removed from deep storage (metadata/deep-storage skew); HDFS name resolution after changing druid.storage.storageDirectory; lost data after NameNode restore; race between retention drop and query loading.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/7d2bcc6c60b2f2da. Report an issue: GitHub.