Failed to close payload for segmente pulled from

Error message

Failed to close payload for segmente pulled from [%s]

What it means

CloudFilesDataSegmentPuller.getSegmentFiles() closes the CloudFiles byte source stream in a finally block; if closeStream() throws IOException it logs this warning. The segment load result is unaffected — this only indicates the HTTP/storage connection was not closed cleanly, which can leak connections over time.

Solutions

Verify network stability between Druid and the Cloud Files endpoint
Reproduce frequency via logs; occasional occurrences can be ignored
Upgrade the storage extension/druid-jersey stack to get connection-pool fixes
Check provider-level connection pool/timeout settings for the rackspace cloudfiles client

Defensive patterns

Strategy: try-catch

Try / catch

try {
  puller.getSegmentFiles(descriptor);
} catch (SegmentLoadingException e) {
  // primary failure handled here
}
// close warning is non-fatal; monitor for frequency to detect connection leaks

Prevention

Keep network paths to Cloud Files stable; check firewall idle-connection drops
Tune the rackspace/cloudfiles HTTP client connection pool and timeouts
Watch for rising counts of this warning — it can indicate connection leaks
Upgrade to a Druid version with stream-close fixes in cloudfiles extension

When it happens

Trigger: closeStream() on the CloudFiles ByteSource fails — typically the underlying HTTP connection to Rackspace was already broken/reset, so closing the stream also errors.

Common situations: Flaky network or Rackspace endpoint resets mid/after transfer; long-running historicals accumulating leaked connections; aggressive firewalls dropping idle connections.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/48ef46fb4c7c19f7. Report an issue: GitHub.

Appendix: source

Thrown at extensions-contrib/cloudfiles-extensions/src/main/java/org/apache/druid/storage/cloudfiles/CloudFilesDataSegmentPuller.java:79

      try {
        FileUtils.deleteDirectory(outDir);
      }
      catch (IOException ioe) {
        log.warn(
            ioe,
            "Failed to remove output directory [%s] for segment pulled from [%s]",
            outDir.getAbsolutePath(),
            path
        );
      }
      throw new SegmentLoadingException(e, e.getMessage());
    }
    finally {
      try {
        byteSource.closeStream();
      }
      catch (IOException ioe) {
        log.warn(ioe, "Failed to close payload for segmente pulled from [%s]", path);
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)