{"record":{"id":"efcfd7fe7fdaa47a","repo":"apache/druid","slug":"failed-to-remove-output-directory-s-for-segment-efcfd7","errorCode":null,"errorMessage":"Failed to remove output directory [%s] for segment pulled from [%s]","messagePattern":"Failed to remove output directory \\[(.+?)\\] for segment pulled from \\[(.+?)\\]","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureDataSegmentPuller.java","lineNumber":89,"sourceCode":"      final String actualBlobPath = AzureUtils.maybeRemoveAzurePathPrefix(blobPath, azureAccountConfig.getBlobStorageEndpoint());\n\n      final ByteSource byteSource = byteSourceFactory.create(containerName, actualBlobPath, azureStorage);\n      final FileUtils.FileCopyResult result = CompressionUtils.unzip(\n          byteSource,\n          outDir,\n          AzureUtils.AZURE_RETRY,\n          false\n      );\n\n      log.info(\"Loaded %d bytes from [%s] to [%s]\", result.size(), actualBlobPath, outDir.getAbsolutePath());\n      return result;\n    }\n    catch (IOException e) {\n      try {\n        FileUtils.deleteDirectory(outDir);\n      }\n      catch (IOException ioe) {\n        log.warn(\n            ioe,\n            \"Failed to remove output directory [%s] for segment pulled from [%s]\",\n            outDir.getAbsolutePath(),\n            blobPath\n        );\n      }\n      throw new SegmentLoadingException(e, e.getMessage());\n    }\n  }\n}\n","sourceCodeStart":71,"sourceCodeEnd":100,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureDataSegmentPuller.java#L71-L100","documentation":"AzureDataSegmentPuller.getSegmentFiles() downloads a segment zip from Azure blob storage and extracts it into outDir. If extraction/read throws IOException, it attempts to delete the partially populated output directory; this warning is logged when that cleanup delete itself fails with a second IOException. The original IOException is still rethrown to the caller.","triggerScenarios":"Segment pull fails mid-extract (corrupt zip blob, truncated download, disk I/O error) AND the subsequent FileUtils.deleteDirectory(outDir) cannot remove the directory (locked files, permissions, full disk).","commonSituations":"Corrupted or partially uploaded segment blob in Azure storage; local disk full on the Druid historical; permission issues on the segment cache directory; files held open by another process.","solutions":["Fix the underlying pull failure first: re-upload or re-download the corrupted segment blob","Check disk space and permissions on druid.segmentCache location","Manually delete the leftover output directory and retry the segment load","Inspect the original IOException (rethrown) for the root cause; the warning is secondary cleanup noise","Restart the process if files are locked and the directory cannot be removed"],"exampleFix":"// before: retrying with a corrupt blob present\n// after: delete the corrupted Azure blob and re-push the segment\nbin/druid segmenttool push --file=segment.zip --bucket=druid --path=wikipedia/2019-04-02T00:00:00.000Z_2019-04-03T00:00:00.000Z/...","handlingStrategy":"retry","validationCode":"// check disk space and permissions on the segment cache dir before pulling\nif (Files.getFileStore(cacheDir).getUsableSpace() < requiredBytes) {\n    throw new IllegalStateException(\"Insufficient disk space for segment cache\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  puller.getSegmentFiles(descriptor, outDir);\n} catch (SegmentLoadingException e) {\n  FileUtils.deleteQuietly(outDir); // clean up what the puller could not\n  retryOrRedownload(e);\n}","preventionTips":["Verify segment blobs are intact after upload (checksums)","Keep druid.segmentCache on a disk with ample free space","Ensure process owns write permissions on the cache directory","Re-push segments that repeatedly fail extraction"],"tags":["azure","storage","io","segment-cleanup"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}