{"record":{"id":"5e27ccb36e8ae49f","repo":"apache/pulsar","slug":"executor-s-should-have-been-shutdown-before-enter","errorCode":null,"errorMessage":"Executor %s should have been shutdown before entering the termination handler.","messagePattern":"Executor (.+?) should have been shutdown before entering the termination handler\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"pulsar-common/src/main/java/org/apache/pulsar/common/util/GracefulExecutorServicesTerminationHandler.java","lineNumber":56,"sourceCode":"@CustomLog\nclass GracefulExecutorServicesTerminationHandler {\n    private static final long SHUTDOWN_THREAD_COMPLETION_TIMEOUT_NANOS = Duration.ofMillis(100L).toNanos();\n    private final List<ExecutorService> executors;\n    private final CompletableFuture<Void> future;\n    private final Duration shutdownTimeout;\n    private final Duration terminationTimeout;\n    private final CountDownLatch shutdownThreadCompletedLatch = new CountDownLatch(1);\n\n    GracefulExecutorServicesTerminationHandler(Duration shutdownTimeout, Duration terminationTimeout,\n                                               List<ExecutorService> executorServices) {\n        this.shutdownTimeout = shutdownTimeout;\n        this.terminationTimeout = terminationTimeout;\n        this.executors = Collections.unmodifiableList(new ArrayList<>(executorServices));\n        this.future = new CompletableFuture<>();\n        log.info().attr(\"executorCount\", executors.size()).log(\"Starting termination handler\");\n        for (ExecutorService executor : executors) {\n            if (!executor.isShutdown()) {\n                throw new IllegalStateException(\n                        String.format(\"Executor %s should have been shutdown before entering the termination handler.\",\n                                executor));\n            }\n        }\n        if (haveExecutorsBeenTerminated()) {\n            markShutdownCompleted();\n        } else {\n            if (shutdownTimeout.isZero() || shutdownTimeout.isNegative()) {\n                terminateExecutors();\n                markShutdownCompleted();\n            } else {\n                Thread shutdownWaitingThread = new Thread(this::awaitShutdown, getClass().getSimpleName());\n                shutdownWaitingThread.setDaemon(false);\n                shutdownWaitingThread.setUncaughtExceptionHandler((thread, exception) -> {\n                  log.error().attr(\"thread\", thread).exception(exception)\n                          .log(\"Uncaught exception in shutdown thread\");\n                });\n                shutdownWaitingThread.start();","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/apache/pulsar/blob/820761864ed8e2a7d2e52dd9763ad2ae117c1395/pulsar-common/src/main/java/org/apache/pulsar/common/util/GracefulExecutorServicesTerminationHandler.java#L38-L74","documentation":"GracefulExecutorServicesTerminationHandler is constructed (via GracefulExecutorServicesShutdown) to await termination of executors that were already shut down. Its constructor validates every ExecutorService with isShutdown() and throws IllegalStateException if any is still running, because awaiting termination of a non-shutdown executor would be a caller bug.","triggerScenarios":"Calling GracefulExecutorServicesShutdown.shutdown(...) (which builds this handler) with an ExecutorService on which shutdown()/shutdownNow() was never called, e.g. mixing graceful shutdown helpers with one manually managed executor.","commonSituations":"Refactoring shutdown code to use GracefulExecutorServicesShutdown while forgetting to call shutdown() on one of the pooled executors; passing executors in the wrong order (handler built before shutdown); Pulsar broker/proxy shutdown code paths during development.","solutions":["Call executor.shutdown() (or shutdownNow()) on every executor before passing it to the graceful shutdown helper.","Use GracefulExecutorServicesShutdown.shutdown(...) as the single entry point so shutdown and termination tracking happen together.","Audit the list of executors being passed and assert each reports isShutdown() before constructing the handler."],"exampleFix":"// before\nGracefulExecutorServicesShutdown.shutdown(Duration.ofSeconds(30), List.of(ioExecutor)); // throws: not shutdown\n// after\nioExecutor.shutdown();\nGracefulExecutorServicesShutdown.shutdown(Duration.ofSeconds(30), List.of(ioExecutor));","handlingStrategy":"validation","validationCode":"static void requireAllShutdown(Collection<ExecutorService> executors) {\n    List<ExecutorService> running = executors.stream()\n            .filter(e -> !e.isShutdown())\n            .collect(Collectors.toList());\n    if (!running.isEmpty()) {\n        running.forEach(ExecutorService::shutdown); // or throw, per policy\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    GracefulExecutorServicesShutdown.shutdown(timeout, executors).get();\n} catch (IllegalStateException e) {\n    if (e.getMessage().contains(\"should have been shutdown\")) {\n        executors.forEach(ExecutorService::shutdownNow);\n    }\n    throw e;\n}","preventionTips":["Always call shutdown()/shutdownNow() on each executor before graceful termination tracking","Centralize executor shutdown in one utility so no executor is missed","Assert isShutdown() on the full list in a debug check during development","Track all executors in a registry at creation time to guarantee complete shutdown lists"],"tags":["executor","shutdown","lifecycle"],"backgroundTag":"executor-not-shutdown","analyzedSha":"820761864ed8e2a7d2e52dd9763ad2ae117c1395","analyzedAt":"2026-09-06T00:14:20.138Z","contentChangedAt":"2026-09-06T00:14:20.138Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}