{"record":{"id":"793e963078969ebc","repo":"apache/druid","slug":"improper-cleanup","errorCode":null,"errorMessage":"Improper cleanup","messagePattern":"Improper cleanup","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/processor/SuperSorter.java","lineNumber":991,"sourceCode":"   *\n   * Note: it is possible for this method to return true even when {@link #activeProcessors} is nonzero. Processors\n   * take some time to exit after the instance becomes \"done\".\n   */\n  @GuardedBy(\"runWorkersLock\")\n  private boolean isAllDone()\n  {\n    return allDone.isDone() || allDone.isCancelled();\n  }\n\n  /**\n   * Cleanup that must happen regardless of success or failure.\n   */\n  @GuardedBy(\"runWorkersLock\")\n  private void cleanUp()\n  {\n    if (!isAllDone() || activeProcessors != 0) {\n      // This condition indicates a logic bug.\n      throw new ISE(\"Improper cleanup\");\n    }\n\n    if (log.isDebugEnabled()) {\n      log.debug(stateString());\n    }\n\n    outputsReadyByLevel.clear();\n    inputBuffer.clear();\n    for (Map.Entry<String, PartitionedOutputChannel> cleanupEntry :\n        levelAndRankToReadableChannelMap.entrySet()) {\n      try {\n        cleanupEntry.getValue().getReadableChannelSupplier().get().close();\n      }\n      catch (IOException e) {\n        throw new UncheckedIOException(\"Unable to close channel for name : \" + cleanupEntry.getKey(), e);\n      }\n    }\n    levelAndRankToReadableChannelMap.clear();","sourceCodeStart":973,"sourceCodeEnd":1009,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/processor/SuperSorter.java#L973-L1009","documentation":"SuperSorter.cleanUp() throws 'Improper cleanup' when cleanup is invoked while the sorter is not fully done (isAllDone() false) or while frame processors are still active (activeProcessors != 0). These two conditions together indicate a lifecycle-management bug: cleanup must only run after all merge work has completed and all processors have been released. It is a defensive assertion protecting against resource-management mistakes in the sorter's state machine.","triggerScenarios":"Calling cleanUp() before every merger processor has finished, or while some input/output processors still hold a reference (activeProcessors > 0); e.g., an error path calls cleanUp() while a worker thread is still running, or close() is invoked twice with interleaved processor completion.","commonSituations":"Hit during query cancellation or error teardown in the MSQ/frame-processor stack when the cancellation path races with ongoing merges; also when developing custom processors that fail to signal completion before triggering sorter cleanup.","solutions":["Ensure the sorter is fully drained (all processors done) before calling cleanUp(); call close()/cancel paths that stop processors and await their completion first.","Verify no double-cleanup: track whether cleanUp() already ran and guard against a second invocation.","Inspect processor exception handling — a processor that threw may never have decremented activeProcessors; make sure failure paths also release processor slots.","Capture stateString() (logged at debug) to see the sorter's level/channel state and identify which processor or channel never finished."],"exampleFix":"// before\nsorter.cleanUp(); // called immediately on error, processors still active\n// after\nif (sorter.isAllDone() && sorter.getActiveProcessors() == 0) {\n  sorter.cleanUp();\n} else {\n  processorManager.cancelAndAwait();\n  sorter.cleanUp();\n}","handlingStrategy":"validation","validationCode":"if (sorter.isAllDone()) {\n  sorter.cleanUp();\n}","typeGuard":null,"tryCatchPattern":"try {\n  sorter.cleanUp();\n} catch (IllegalStateException e) {\n  if (\"Improper cleanup\".equals(e.getMessage())) {\n    log.warn(e, \"Sorter cleanup raced with active processors; state=%s\", sorter.stateString());\n  } else {\n    throw e;\n  }\n}","preventionTips":["Only call cleanUp() after all processors have signaled completion.","Ensure failure paths of processors also decrement activeProcessors.","Make cleanup single-shot; guard against double invocation.","Enable debug logging of stateString() in test environments to catch lifecycle bugs early."],"tags":["internal-state","resource-cleanup","lifecycle"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}