{"record":{"id":"a9ba515d308ba637","repo":"apache/druid","slug":"failed-to-remove-output-directory-s-for-segment-a9ba51","errorCode":null,"errorMessage":"Failed to remove output directory [%s] for segment pulled from [%s]","messagePattern":"Failed to remove output directory \\[(.+?)\\] for segment pulled from \\[(.+?)\\]","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleDataSegmentPuller.java","lineNumber":71,"sourceCode":"    try {\n      FileUtils.mkdirp(outDir);\n\n      final GoogleByteSource byteSource = new GoogleByteSource(storage, bucket, path);\n      final FileUtils.FileCopyResult result = CompressionUtils.unzip(\n          byteSource,\n          outDir,\n          GoogleUtils::isRetryable,\n          false\n      );\n      LOG.info(\"Loaded %d bytes from [%s] to [%s]\", result.size(), path, outDir.getAbsolutePath());\n      return result;\n    }\n    catch (Exception e) {\n      try {\n        FileUtils.deleteDirectory(outDir);\n      }\n      catch (IOException ioe) {\n        LOG.warn(\n            ioe,\n            \"Failed to remove output directory [%s] for segment pulled from [%s]\",\n            outDir.getAbsolutePath(),\n            path\n        );\n      }\n      throw new SegmentLoadingException(e, e.getMessage());\n    }\n  }\n\n  @Override\n  public InputStream getInputStream(URI uri) throws IOException\n  {\n    String path = StringUtils.maybeRemoveLeadingSlash(uri.getPath());\n    return storage.getInputStream(uri.getHost() != null ? uri.getHost() : uri.getAuthority(), path);\n  }\n\n  @Override","sourceCodeStart":53,"sourceCodeEnd":89,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleDataSegmentPuller.java#L53-L89","documentation":"GoogleDataSegmentPuller.getSegmentFiles() downloads a segment from GCS into a temp output directory. If any exception occurs during the pull, it attempts to clean up that directory with FileUtils.deleteDirectory; if that cleanup itself fails with IOException it logs this warning. The original pull error is still propagated to the caller; this message only signals leftover temp files.","triggerScenarios":"A segment pull from GCS fails (e.g. network error, missing object, decompression failure) AND the temp output directory cannot be deleted (file lock, permission issue, disk problem on the local host).","commonSituations":"GCS outage or wrong permissions on the segment path combined with a local disk that won't allow deletion (read-only mount, running as non-root with root-owned temp dirs, NFS stale handles). Also common in tests that force pull failures while the outDir is not deletable.","solutions":["Fix the underlying pull failure first — check GCS permissions/bucket/object existence; this warning is secondary","Manually delete leftover temp directories (segment pulling dirs under the druid temp location) to reclaim disk space","Check filesystem permissions and disk health on the Druid historical/middleManager host","If recurring in tests, assert the original exception rather than relying on directory cleanup"],"exampleFix":"// before\ncatch (Exception e) {\n  try { FileUtils.deleteDirectory(outDir); }\n  catch (IOException ioe) { LOG.warn(ioe, \"Failed to remove output directory [%s] ...\", outDir.getAbsolutePath(), path); }\n  throw e;\n}\n// after\ncatch (Exception e) {\n  try {\n    FileUtils.deleteDirectory(outDir);\n  } catch (IOException ioe) {\n    LOG.warn(ioe, \"Failed to remove output directory [%s] for segment pulled from [%s]\", outDir.getAbsolutePath(), path);\n    outDir.deleteOnExit();\n  }\n  throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  puller.getSegmentFiles(path, outDir);\n} catch (SegmentLoadingException e) {\n  // handle the real pull failure; temp-dir cleanup warning is secondary\n  LOG.error(\"Segment pull failed from [%s]: %s\", path, e.getMessage());\n}","preventionTips":["Monitor local disk health and permissions on Druid nodes","Ensure the Druid process user owns its temp directories","Alert on the underlying SegmentLoadingException, not this cleanup warning"],"tags":["google-cloud-storage","segment-pull","cleanup","ioexception","druid"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}