{"record":{"id":"b86d0aedda594cff","repo":"apache/druid","slug":"failed-to-run-deferred-eviction-of-container-d-f","errorCode":null,"errorMessage":"Failed to run deferred eviction of container[%d] for [%s]","messagePattern":"Failed to run deferred eviction of container\\[(.+?)\\] for \\[(.+?)\\]","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"processing/src/main/java/org/apache/druid/segment/file/PartialSegmentFileMapperV10.java","lineNumber":1031,"sourceCode":"    boolean evictNow = false;\n    containerLocks[containerIndex].lock();\n    try {\n      containerFetchesInFlight[containerIndex]--;\n      if (containerFetchesInFlight[containerIndex] == 0 && containerEvictionPending[containerIndex]) {\n        containerEvictionPending[containerIndex] = false;\n        evictNow = true;\n      }\n    }\n    finally {\n      containerLocks[containerIndex].unlock();\n    }\n    if (evictNow) {\n      // Runs from fetchRun's finally, so it must not throw over whatever brought us here.\n      try {\n        evictContainer(containerIndex);\n      }\n      catch (Throwable t) {\n        LOG.warn(t, \"Failed to run deferred eviction of container[%d] for [%s]\", containerIndex, targetFilename);\n      }\n    }\n  }\n\n  /**\n   * Reverse of {@link #initializeContainer(int)}: unmap the in-memory view of the container, delete the local\n   * container file, and clear the bitmap bits + {@link #downloadedFiles} entries for every internal file that lived\n   * in this container.\n   * <p>\n   * <b>Concurrency contract.</b> No concurrent {@link #mapFile} may be in flight for any file in this container;\n   * that is enforced one layer up by the cache-entry refcount, since {@code PartialSegmentBundleCacheEntry} evicts\n   * from its {@code doActualUnmount} callback, which fires only after every reference acquired via\n   * {@code acquireReference()} has been closed. Bypassing that gate is dangerous: {@link ByteBufferUtils#unmap} frees\n   * the off-heap mapping, so a {@link ByteBuffer#slice} from a concurrent reader is a JVM SIGSEGV, not a recoverable\n   * error. An in-flight {@link #fetchFiles}/{@link #fetchRun} is handled here instead of by the caller: the eviction\n   * is deferred to whichever fetch finishes last, because callers can hold the storage location's write lock and must\n   * not block on a deep-storage read.\n   * <p>","sourceCodeStart":1013,"sourceCodeEnd":1049,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/segment/file/PartialSegmentFileMapperV10.java#L1013-L1049","documentation":"During deferred eviction (unmapping and deleting the local container file) run in fetchRun's finally block, evictContainer threw. Because this runs during cleanup, the exception is only logged as a warning and swallowed so it does not mask the original fetch outcome. Failure here can leave a container view mapped or its local file present.","triggerScenarios":"evictContainer (which unmaps the in-memory view and deletes local files) throws during deferred eviction after a fetch, e.g. munmap failure, file lock, or I/O error deleting the local file.","commonSituations":"Disk issues on the segment cache directory, concurrent access to the container file, resource exhaustion preventing unmapping on the host.","solutions":["Check the logged Throwable for the eviction root cause (I/O vs mapping failure)","Verify disk and permissions on the local segment cache directory","Restart the node if container mappings leak (check memory-mapped regions)","Retry the fetch; eviction is retried on subsequent runs"],"exampleFix":"null","handlingStrategy":"fallback","validationCode":"// Ensure cache directory exists and is writable before enabling containerized fetch\nFiles.isDirectory(cacheDir) && Files.isWritable(cacheDir);","typeGuard":"null","tryCatchPattern":"// Deferred eviction is non-throwing by design; monitor WARN logs and rely on retry fetch\n// Treat as best-effort cleanup — no caller action required","preventionTips":["Monitor disk space and permissions on the local segment cache directory","Watch for leaked mappings after WARN events and restart if memory grows","Keep the node's local storage healthy"],"tags":["segment-cache","eviction","file"],"backgroundTag":"file-write-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}