{"record":{"id":"dda8bf18fc8b44e6","repo":"apache/druid","slug":"s-is-a-child-directory-skipping","errorCode":null,"errorMessage":"[%s] is a child directory, skipping","messagePattern":"\\[(.+?)\\] is a child directory, skipping","errorType":"console","errorClass":null,"httpStatus":null,"severity":"info","filePath":"extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java","lineNumber":227,"sourceCode":"      final FileSystem fs = path.getFileSystem(config);\n      if (fs.getFileStatus(path).isDirectory()) {\n\n        // --------    directory     ---------\n        try {\n          return RetryUtils.retry(\n              () -> {\n                if (!fs.exists(path)) {\n                  throw new SegmentLoadingException(\"No files found at [%s]\", path.toString());\n                }\n\n                final RemoteIterator<LocatedFileStatus> children = fs.listFiles(path, false);\n                final FileUtils.FileCopyResult result = new FileUtils.FileCopyResult();\n                while (children.hasNext()) {\n                  final LocatedFileStatus child = children.next();\n                  final Path childPath = child.getPath();\n                  final String fname = childPath.getName();\n                  if (fs.getFileStatus(childPath).isDirectory()) {\n                    log.warn(\"[%s] is a child directory, skipping\", childPath.toString());\n                  } else {\n                    final File outFile = new File(outDir, fname);\n                    try (final FSDataInputStream in = fs.open(childPath)) {\n                      NativeIO.chunkedCopy(in, outFile);\n                    }\n                    result.addFile(outFile);\n                  }\n                }\n                log.info(\n                    \"Copied %d bytes from [%s] to [%s]\",\n                    result.size(),\n                    path.toString(),\n                    outDir.getAbsolutePath()\n                );\n                return result;\n              },\n              shouldRetryPredicate(),\n              DEFAULT_RETRY_COUNT","sourceCodeStart":209,"sourceCodeEnd":245,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/hdfs-storage/src/main/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPuller.java#L209-L245","documentation":"HdfsDataSegmentPuller.getSegmentFiles() copies segment files from HDFS into a local output directory. When a child entry in the segment directory is itself a directory, it cannot be copied as a segment file, so it logs this warning and skips it. It is informational — the zip's expected files (index, metadata, etc.) are still copied.","triggerScenarios":"A segment directory on HDFS contains subdirectories, e.g. leftover partial pushes, manually copied directories, or a corrupt push that nested files incorrectly.","commonSituations":"Interrupted segment pushes leaving temp subdirectories, users manually extracting/re-zipping segments on HDFS, or debugging pull failures by unpacking zips into the segment location.","solutions":["Inspect the HDFS segment directory with `hdfs dfs -ls` and remove unexpected subdirectories","Re-push the segment (e.g. reset segments / re-run the task) so the directory contains only index.zip and companions","No action if segments load fine — the skipped directory is harmless","Fix whatever process created nested directories (usually a custom pusher or manual copy)"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// before pulling, ensure the segment dir holds only files\nfor (FileStatus st : fs.listStatus(segmentDir)) {\n  if (st.isDirectory()) { /* clean up or re-push segment */ }\n}","typeGuard":null,"tryCatchPattern":"try {\n  FileSegmentPuller.FilesProvider result = puller.getSegmentFiles(path, outDir);\n} catch (SegmentLoadingException e) {\n  // handle; note child directories are skipped with a warning only\n}","preventionTips":["Avoid manually unpacking files into segment directories on HDFS","Ensure pushes complete atomically so no temp subdirs remain","Re-push segments whose directories contain unexpected children"],"tags":["hdfs","segment-pull","deep-storage","druid"],"backgroundTag":"path-is-not-a-directory","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}