apache/druid · warning

Failed to remove output directory

Error message

Failed to remove output directory [%s] for segment pulled from [%s]

What it means

CloudFilesDataSegmentPuller.getSegmentFiles() pulls a segment from Rackspace Cloud Files into a temp dir; on pull failure it attempts FileUtils.deleteDirectory(outDir) and logs this warning if that cleanup throws IOException. The real SegmentLoadingException is rethrown to the caller; this warning signals leftover temp directories.

Solutions

1. Resolve the pull failure: verify Cloud Files credentials, container, and region config
2. Remove leftover temp dirs under the segment cache location manually
3. Check permissions and mount health (not read-only) for the cache directory
4. Ensure the filesystem supports reliable recursive delete (avoid NFS if persistent)

// preflight Cloud Files access
Container container = cfClient.getContainer(name); // fails fast on auth/region issues

try {
  puller.getSegmentFiles(descriptor);
} catch (SegmentLoadingException e) {
  log.warn(e, "Cloud Files pull failed; leftover temp dir possible");
}

When it happens

Trigger: Segment download from Cloud Files fails (auth failure, missing container/object, corrupt archive) and the subsequent temp-dir deletion fails due to open file handles, permissions, or filesystem errors.

Common situations: Expired or wrong Rackspace API credentials; container name/region misconfig; segment cache on a shared/NFS filesystem; disk full or read-only mount during cleanup.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

• Could not delete cache file at
• Failed to close payload for segmente pulled from
• Failed to remove output directory
• Asked to load path[ ], but it doesn't exist.
• Do not know how to handle file type at

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/771b7d478bfe3a25. Report an issue: GitHub.

Appendix: source

Thrown at extensions-contrib/cloudfiles-extensions/src/main/java/org/apache/druid/storage/cloudfiles/CloudFilesDataSegmentPuller.java:65

    CloudFilesObjectApiProxy objectApi = new CloudFilesObjectApiProxy(cloudFilesApi, region, container);
    final CloudFilesByteSource byteSource = new CloudFilesByteSource(objectApi, path);

    try {
      final FileUtils.FileCopyResult result = CompressionUtils.unzip(
          byteSource,
          outDir,
          CloudFilesUtils.CLOUDFILESRETRY,
          false
      );
      log.info("Loaded %d bytes from [%s] to [%s]", result.size(), path, outDir.getAbsolutePath());
      return result;
    }
    catch (Exception e) {
      try {
        FileUtils.deleteDirectory(outDir);
      }
      catch (IOException ioe) {
        log.warn(
            ioe,
            "Failed to remove output directory [%s] for segment pulled from [%s]",
            outDir.getAbsolutePath(),
            path
        );
      }
      throw new SegmentLoadingException(e, e.getMessage());
    }
    finally {
      try {
        byteSource.closeStream();
      }
      catch (IOException ioe) {
        log.warn(ioe, "Failed to close payload for segmente pulled from [%s]", path);
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)