{"record":{"id":"bfa3332a91643a55","repo":"apache/druid","slug":"outdir-s-must-exist-and-be-a-directory","errorCode":null,"errorMessage":"outDir[%s] must exist and be a directory","messagePattern":"outDir\\[(.+?)\\] must exist and be a directory","errorType":"validation","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/utils/CompressionUtils.java","lineNumber":362,"sourceCode":"\n      // Copy file content to dataOut\n      try (FileInputStream fileIn = new FileInputStream(file)) {\n        ByteStreams.copy(fileIn, dataOut);\n      }\n    }\n\n    dataOut.flush();\n    lz4Out.finish();\n    return totalSize;\n  }\n\n  /**\n   * Decompresses LZ4-compressed directory archive\n   */\n  public static FileUtils.FileCopyResult lz4DecompressDirectory(InputStream in, File outDir) throws IOException\n  {\n    if (!(outDir.exists() && outDir.isDirectory())) {\n      throw new ISE(\"outDir[%s] must exist and be a directory\", outDir);\n    }\n\n    final LZ4BlockInputStream lz4In = new LZ4BlockInputStream(in);\n    final DataInputStream dataIn = new DataInputStream(lz4In);\n\n    final int fileCount = dataIn.readInt();\n    final FileUtils.FileCopyResult result = new FileUtils.FileCopyResult();\n\n    for (int i = 0; i < fileCount; i++) {\n      final int fileNameLength = dataIn.readInt();\n      final byte[] fileNameBytes = new byte[fileNameLength];\n      dataIn.readFully(fileNameBytes);\n      final String fileName = new String(fileNameBytes, StandardCharsets.UTF_8);\n\n      final long fileSize = dataIn.readLong();\n\n      // Write to file\n      final File outFile = new File(outDir, fileName);","sourceCodeStart":344,"sourceCodeEnd":380,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/utils/CompressionUtils.java#L344-L380","documentation":"Precondition guard in the LZ4 directory decompression path: lz4DecompressDirectory streams an LZ4-compressed directory archive into an output directory that must already exist on disk, because the routine writes entries relative to it and never creates the root itself. It fires when callers (decompressDirectory) pass an outDir that does not exist or is a regular file rather than a directory; create the target directory before decompressing.","triggerScenarios":"Calling decompressDirectory (LZ4 mode) with an outDir File that was never mkdir'd, was deleted earlier in the flow, or is a plain file.","commonSituations":"Fresh worker nodes where the local segment cache dir hasn't been created yet, typo'd outdir path in task config, or code that assumes decompress creates the directory.","solutions":["Create the directory before calling: `Files.mkdirs(outDir)` or `outDir.mkdirs()` and verify `outDir.isDirectory()`.","Check the configured path for typos and ensure the Druid process user can create/write there.","If the directory was deleted by cleanup, recreate it before decompression."],"exampleFix":"// before\nCompressionUtils.lz4DecompressDirectory(in, new File(\"/tmp/out\"));\n// after\nfinal File outDir = new File(\"/tmp/out\");\nif (!outDir.exists() && !outDir.mkdirs()) {\n  throw new IOException(\"Failed to create outDir: \" + outDir);\n}\nCompressionUtils.lz4DecompressDirectory(in, outDir);","handlingStrategy":"validation","validationCode":"final File outDir = new File(dest);\nif (!outDir.isDirectory() && !outDir.mkdirs()) {\n  throw new IOException(\"Cannot create output directory: \" + outDir);\n}","typeGuard":"static boolean isValidOutDir(File f) {\n  return f != null && f.exists() && f.isDirectory();\n}","tryCatchPattern":"try {\n  CompressionUtils.lz4DecompressDirectory(in, outDir);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"must exist and be a directory\")) {\n    outDir.mkdirs();\n    // retry once\n  }\n}","preventionTips":["Always mkdirs() the destination before decompression","Verify isDirectory() after mkdirs","Check process write permissions on the parent path","Don't reuse a file path where a directory is expected"],"tags":["io","filesystem","compression"],"backgroundTag":"directory-not-found","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}