{"record":{"id":"5caaa26a1787461c","repo":"apache/cassandra","slug":"unable-to-terminate-fast-tasks-within-1-minute","errorCode":null,"errorMessage":"Unable to terminate fast tasks within 1 minute.","messagePattern":"Unable to terminate fast tasks within 1 minute\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/service/StorageService.java","lineNumber":766,"sourceCode":"        {\n            throw new AssertionError(e);\n        }\n\n        // daemon threads, like our executors', continue to run while shutdown hooks are invoked\n        drainOnShutdown = NamedThreadFactory.createThread(new WrappedRunnable()\n        {\n            @Override\n            public void runMayThrow() throws InterruptedException, ExecutionException, IOException\n            {\n                drain(true);\n                try\n                {\n                    ExecutorUtils.shutdownNowAndWait(1, MINUTES, ScheduledExecutors.scheduledFastTasks);\n                    logger.info(\"Cassandra shutdown complete\");\n                }\n                catch (Throwable t)\n                {\n                    logger.warn(\"Unable to terminate fast tasks within 1 minute.\", t);\n                }\n                finally\n                {\n                    LoggingSupportFactory.getLoggingSupport().onShutdown();\n                }\n            }\n        }, \"StorageServiceShutdownHook\");\n        Runtime.getRuntime().addShutdownHook(drainOnShutdown);\n\n        Schema.instance.saveSystemKeyspace();\n        DatabaseDescriptor.getInternodeAuthenticator().setupInternode();\n\n        if (ClusterMetadataService.state() == ClusterMetadataService.State.GOSSIP)\n        {\n            // register listener before starting gossiper to avoid missing messages\n            Gossiper.instance.register(new GossipCMSListener());\n        }\n        sstablesTracker.register((notification, o) -> {","sourceCodeStart":748,"sourceCodeEnd":784,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/service/StorageService.java#L748-L784","documentation":"WARN logged in StorageService.runMayThrow's shutdown hook when ExecutorUtils.shutdownNowAndWait(1, MINUTES, ScheduledExecutors.scheduledFastTasks) does not finish within its 1-minute timeout. Some fast scheduled tasks refused to terminate promptly at JVM shutdown, so shutdown proceeds with those tasks still running or interrupted; the original Throwable is attached.","triggerScenarios":"JVM shutdown hook runs (SIGTERM / daemon.decommission) and scheduledFastTasks executor fails to terminate within the 1-minute wait — typically tasks blocked on slow I/O, long compaction-adjacent work, or deadlocks.","commonSituations":"Graceful shutdown under heavy load (draining, large memtable flushes), container stop timeouts, hung worker threads blocking the scheduled executor.","solutions":["Inspect the attached stack trace (t) to identify the stuck task class.","Increase shutdown grace period (e.g. container stop timeout) and retry shutdown.","Take a thread dump during shutdown (jstack) to find blocking code.","If a specific task consistently hangs, report/investigate that executor's task; workaround by stopping dependent subsystems first ('nodetool drain')."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// shutdown hook already catches Throwable; treat WARN as signal to inspect hung executors\ntry {\n    ExecutorUtils.shutdownNowAndWait(1, MINUTES, scheduledFastTasks);\n} catch (Throwable t) {\n    logger.warn(\"Unable to terminate fast tasks within 1 minute.\", t);\n}","preventionTips":["Take thread dumps during slow shutdowns to find hung tasks.","Increase container/process stop timeout beyond task worst case.","Run 'nodetool drain' before shutdown to quiesce work."],"tags":["cassandra","shutdown","executor","timeout"],"backgroundTag":"request-timeout","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}