{"record":{"id":"8e1b8fe415aacc39","repo":"apache/druid","slug":"unexpected-subdirectory-s-8e1b8f","errorCode":null,"errorMessage":"Unexpected subdirectory [%s]","messagePattern":"Unexpected subdirectory \\[(.+?)\\]","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPusher.java","lineNumber":146,"sourceCode":"  private DataSegment pushNoZip(final File inDir, final File outDir, final DataSegment baseSegment) throws IOException\n  {\n    final File tmpSegmentDir = new File(config.getStorageDirectory(), makeIntermediateDir());\n    FileUtils.mkdirp(tmpSegmentDir);\n\n    try {\n      final File[] files = inDir.listFiles();\n      if (files == null) {\n        throw new IOE(\"Cannot list directory [%s]\", inDir);\n      }\n\n      long size = 0;\n      for (final File file : files) {\n        if (file.isFile()) {\n          size += file.length();\n          FileUtils.linkOrCopy(file, new File(tmpSegmentDir, file.getName()));\n        } else {\n          // Segment directories are expected to be flat.\n          throw new IOE(\"Unexpected subdirectory [%s]\", file.getName());\n        }\n      }\n\n      final File segmentDir = new File(outDir, INDEX_DIR);\n      FileUtils.mkdirp(outDir);\n\n      try {\n        Files.move(tmpSegmentDir.toPath(), segmentDir.toPath(), StandardCopyOption.ATOMIC_MOVE);\n      }\n      catch (IOException e) {\n        if (segmentDir.exists()) {\n          // Move old directory out of the way, then try again. This makes the latest push win when we push to the\n          // same directory twice, so behavior is compatible with the zip style of pushing.\n          Files.move(\n              segmentDir.toPath(),\n              new File(outDir, StringUtils.format(\"%s_old_%s\", INDEX_DIR, UUID.randomUUID())).toPath(),\n              StandardCopyOption.ATOMIC_MOVE\n          );","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/server/src/main/java/org/apache/druid/segment/loading/LocalDataSegmentPusher.java#L128-L164","documentation":"LocalDataSegmentPusher.pushNoZip throws IOException when it encounters a subdirectory inside the segment input directory. Segment directories are expected to be flat (index.drd, data files at top level); a nested directory means the segment layout is not what the no-zip pusher supports, so it refuses to push rather than producing a broken segment.","triggerScenarios":"pushToPath -> pushNoZip iterates inDir.listFiles() and finds file.isDirectory() true — a nested folder in the segment output.","commonSituations":"Custom extension or pre-processing step writing extra subdirectories (logs, smolder temp dirs) into the segment dir; users manually placing files in the task output; mixing uncompressed push with layouts produced by other tools.","solutions":["Remove unexpected subdirectories from the segment output directory and re-push.","Ensure the ingestion task produces a flat segment directory (only files at top level).","Use zip push mode (druid.storage.type / pusher default) if nested layouts are unavoidable.","Check custom plugins/external scripts that write into the task work directory."],"exampleFix":"// before: extra subdir left in segment dir\n/segment-dir/index.drd\n/segment-dir/data/000.smoosh  <-- 'data' is a subdirectory\n// after: flatten\n/segment-dir/index.drd\n/segment-dir/000.smoosh","handlingStrategy":"validation","validationCode":"File[] files = inDir.listFiles();\nif (files != null) {\n  for (File f : files) {\n    if (f.isDirectory()) throw new IllegalStateException(\"nested dir in segment output, flatten first: \" + f);\n  }\n}","typeGuard":"boolean isFlatSegmentDir(File dir) {\n  File[] files = dir.listFiles();\n  if (files == null) return false;\n  return java.util.Arrays.stream(files).allMatch(File::isFile);\n}","tryCatchPattern":"try {\n  pusher.push(segment, outDir, false);\n} catch (IOException e) {\n  log.error(e, \"segment dir layout invalid; flatten and retry\");\n  throw e;\n}","preventionTips":["Keep ingestion output flat: only files directly inside the segment directory.","Keep logs/temp files outside the task's segment output directory.","Prefer zip push mode when produced layouts are not guaranteed flat.","Audit custom plugins for writes into the segment directory."],"tags":["segment-push","invalid-layout","local-deep-storage"],"backgroundTag":"invalid-argument-value","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}