{"record":{"id":"4a5d4e173924be6c","repo":"apache/druid","slug":"session-keeper-service-did-not-terminate-in-time","errorCode":null,"errorMessage":"Session keeper service did not terminate in time","messagePattern":"Session keeper service did not terminate in time","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"extensions-contrib/consul-extensions/src/main/java/org/apache/druid/consul/discovery/ConsulLeaderSelector.java","lineNumber":209,"sourceCode":"      }\n\n      if (executorService != null) {\n        executorService.shutdownNow();\n        try {\n          if (!executorService.awaitTermination(5, TimeUnit.SECONDS)) {\n            LOGGER.warn(\"Leader selector executor did not terminate in time\");\n          }\n        }\n        catch (InterruptedException e) {\n          Thread.currentThread().interrupt();\n        }\n      }\n\n      if (sessionKeeperService != null) {\n        sessionKeeperService.shutdownNow();\n        try {\n          if (!sessionKeeperService.awaitTermination(5, TimeUnit.SECONDS)) {\n            LOGGER.warn(\"Session keeper service did not terminate in time\");\n          }\n        }\n        catch (InterruptedException e) {\n          Thread.currentThread().interrupt();\n        }\n      }\n    }\n    finally {\n      lifecycleLock.exitStop();\n    }\n  }\n\n  private void startLeaderElection()\n  {\n    executorService.submit(this::leaderElectionLoop);\n    sessionKeeperService.submit(this::sessionKeeperLoop);\n  }\n","sourceCodeStart":191,"sourceCodeEnd":227,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-contrib/consul-extensions/src/main/java/org/apache/druid/consul/discovery/ConsulLeaderSelector.java#L191-L227","documentation":"Warning logged during unregisterListener() when the session-keeper executor fails to terminate within the 5-second awaitTermination window after shutdownNow(). The keeper loop renews the Consul session on a schedule; if it is mid-renewal in a slow RPC or sleeping, it may outlive the wait, leaving a lingering thread that exits on its next interruptible point.","triggerScenarios":"Calling unregisterListener() while sessionKeeperLoop is blocked in consulClient.renewSession() against a slow/unreachable Consul, or in the scheduled-delay before its next renewal run.","commonSituations":"Network partitions during service shutdown, Consul agent stalls, or healthCheckInterval large enough that the scheduled renewal task sits queued beyond the 5s wait.","solutions":["Verify Consul connectivity/latency at shutdown time; fix agent or network stalls.","Lower healthCheckInterval so scheduled renewal tasks are short and frequent.","Accept the warning if benign: shutdownNow() interrupted the task and it will terminate at the next interruptible call."],"exampleFix":"// before\ndruid.discovery.consul.service.healthCheckInterval=PT2M\n// after\ndruid.discovery.consul.service.healthCheckInterval=PT15S","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  selector.unregisterListener();\n} catch (RuntimeException e) {\n  LOGGER.warn(e, \"Leader selector stop reported issues; keeper thread will exit at next interrupt point\");\n}","preventionTips":["Keep healthCheckInterval short relative to TTL","Avoid shutting down while Consul is known to be unreachable","Gracefully destroy sessions before executor shutdown where possible"],"tags":["shutdown","executor","timeout","consul","session-renewal"],"backgroundTag":"request-timeout","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}