What causes "[%s] is a child directory, skipping" in apache/druid?

Inspect the HDFS segment directory with `hdfs dfs -ls` and remove unexpected subdirectories

[%s] is a child directory, skipping

Error message

[%s] is a child directory, skipping

What it means

HdfsDataSegmentPuller.getSegmentFiles() copies segment files from HDFS into a local output directory. When a child entry in the segment directory is itself a directory, it cannot be copied as a segment file, so it logs this warning and skips it. It is informational — the zip's expected files (index, metadata, etc.) are still copied.

Source

Thrown at extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java:227

      final FileSystem fs = path.getFileSystem(config);
      if (fs.getFileStatus(path).isDirectory()) {

        // --------    directory     ---------
        try {
          return RetryUtils.retry(
              () -> {
                if (!fs.exists(path)) {
                  throw new SegmentLoadingException("No files found at [%s]", path.toString());
                }

                final RemoteIterator<LocatedFileStatus> children = fs.listFiles(path, false);
                final FileUtils.FileCopyResult result = new FileUtils.FileCopyResult();
                while (children.hasNext()) {
                  final LocatedFileStatus child = children.next();
                  final Path childPath = child.getPath();
                  final String fname = childPath.getName();
                  if (fs.getFileStatus(childPath).isDirectory()) {
                    log.warn("[%s] is a child directory, skipping", childPath.toString());
                  } else {
                    final File outFile = new File(outDir, fname);
                    try (final FSDataInputStream in = fs.open(childPath)) {
                      NativeIO.chunkedCopy(in, outFile);
                    }
                    result.addFile(outFile);
                  }
                }
                log.info(
                    "Copied %d bytes from [%s] to [%s]",
                    result.size(),
                    path.toString(),
                    outDir.getAbsolutePath()
                );
                return result;
              },
              shouldRetryPredicate(),
              DEFAULT_RETRY_COUNT

View on GitHub (pinned to 9b90983fd2)

Solutions

Inspect the HDFS segment directory with `hdfs dfs -ls` and remove unexpected subdirectories
Re-push the segment (e.g. reset segments / re-run the task) so the directory contains only index.zip and companions
No action if segments load fine — the skipped directory is harmless
Fix whatever process created nested directories (usually a custom pusher or manual copy)

Defensive patterns

Strategy: fallback

Validate before calling

// before pulling, ensure the segment dir holds only files
for (FileStatus st : fs.listStatus(segmentDir)) {
  if (st.isDirectory()) { /* clean up or re-push segment */ }
}

Try / catch

try {
  FileSegmentPuller.FilesProvider result = puller.getSegmentFiles(path, outDir);
} catch (SegmentLoadingException e) {
  // handle; note child directories are skipped with a warning only
}

Prevention

Avoid manually unpacking files into segment directories on HDFS
Ensure pushes complete atomically so no temp subdirs remain
Re-push segments whose directories contain unexpected children

When it happens

Trigger: A segment directory on HDFS contains subdirectories, e.g. leftover partial pushes, manually copied directories, or a corrupt push that nested files incorrectly.

Common situations: Interrupted segment pushes leaving temp subdirectories, users manually extracting/re-zipping segments on HDFS, or debugging pull failures by unpacking zips into the segment location.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/dda8bf18fc8b44e6. Report an issue: GitHub.