{"record":{"id":"ac6f97bcd129f9b6","repo":"apache/cassandra","slug":"scheduledthreadpoolexecutor-has-shut-down","errorCode":null,"errorMessage":"ScheduledThreadPoolExecutor has shut down.","messagePattern":"ScheduledThreadPoolExecutor has shut down\\.","errorType":"exception","errorClass":"RejectedExecutionException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/concurrent/ScheduledThreadPoolExecutorPlus.java","lineNumber":63,"sourceCode":" *\n * Catches exceptions during Task execution so that they don't suppress subsequent invocations of the task.\n *\n * Finally, there is a special rejected execution handler for tasks rejected during the shutdown hook.\n *  - For fire and forget tasks (like ref tidy) we can safely ignore the exceptions.\n *  - For any callers that care to know their task was rejected we cancel passed task.\n */\npublic class ScheduledThreadPoolExecutorPlus extends ScheduledThreadPoolExecutor implements ScheduledExecutorPlus\n{\n    private static final Logger logger = LoggerFactory.getLogger(ScheduledThreadPoolExecutorPlus.class);\n    private static final TaskFactory taskFactory = TaskFactory.standard();\n\n    public static final RejectedExecutionHandler rejectedExecutionHandler = (task, executor) ->\n    {\n        if (executor.isShutdown())\n        {\n            // TODO: this sequence of events seems poorly thought out\n            if (!StorageService.instance.isShutdown())\n                throw new RejectedExecutionException(\"ScheduledThreadPoolExecutor has shut down.\");\n\n            //Give some notification to the caller the task isn't going to run\n            if (task instanceof java.util.concurrent.Future)\n                ((java.util.concurrent.Future<?>) task).cancel(false);\n\n            logger.debug(\"ScheduledThreadPoolExecutor has shut down as part of C* shutdown\");\n        }\n        else\n        {\n            throw new AssertionError(\"Unknown rejection of ScheduledThreadPoolExecutor task\");\n        }\n    };\n\n    ScheduledThreadPoolExecutorPlus(NamedThreadFactory threadFactory)\n    {\n        super(1, threadFactory);\n        setRejectedExecutionHandler(rejectedExecutionHandler);\n    }","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/concurrent/ScheduledThreadPoolExecutorPlus.java#L45-L81","documentation":"When a task is submitted to a ScheduledThreadPoolExecutorPlus that has already been shut down, the rejected-execution handler throws RejectedExecutionException — but only if the whole Cassandra node (StorageService) is NOT shutting down. If the node itself is going down, the rejection is treated as expected: the task is cancelled and only logged. This distinguishes an application bug (submitting after executor shutdown) from normal node shutdown.","triggerScenarios":"Submitting a scheduled task after shutdown() was called on the executor while StorageService.isShutdown() is false; lifecycle races where a component keeps posting periodic tasks to an executor that another component already shut down.","commonSituations":"Component ordering bugs during partial shutdown (one subsystem stops its executor while others still submit); restarting a service in-process after shutdown without recreating the executor; tests that shut down executors but leave scheduled jobs registered.","solutions":["Ensure the executor is only shut down when the owning component stops submitting to it; coordinate lifecycle so shutdown happens last","Recreate the executor (or never shut it down) if submissions continue — ScheduledThreadPoolExecutorPlus instances are typically global and node-lifecycle-scoped","Catch RejectedExecutionException at the submit site and treat it as 'executor stopped' for the caller","If seen during node shutdown, verify StorageService.isShutdown() returns true; if the node is mid-shutdown this message is expected noise"],"exampleFix":"// before\nscheduledExecutor.shutdown();\nscheduledExecutor.schedule(task, 1, SECONDS); // RejectedExecutionException\n// after\nif (!scheduledExecutor.isShutdown())\n    scheduledExecutor.schedule(task, 1, SECONDS);","handlingStrategy":"try-catch","validationCode":"if (executor.isShutdown())\n    throw new IllegalStateException(\"cannot schedule on shut-down executor\");","typeGuard":"static boolean canSchedule(ScheduledThreadPoolExecutorPlus e)\n{\n    return !e.isShutdown();\n}","tryCatchPattern":"try\n{\n    executor.schedule(task, delay, unit);\n}\ncatch (RejectedExecutionException e)\n{\n    // executor shut down; drop the task or route it elsewhere\n    logger.info(\"Task dropped: executor shut down\");\n}","preventionTips":["Align executor shutdown with the end of all producers using it","Check isShutdown() before scheduling in racy lifecycle code","Treat these executors as node-scoped singletons and never shut them down mid-operation","Handle RejectedExecutionException at periodic-task registration sites"],"tags":["rejected-execution","executor","lifecycle"],"backgroundTag":"invalid-state-transition","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}