{"record":{"id":"531ed5ce6247e4e7","repo":"apache/cassandra","slug":"unable-to-start-s-because-the-node-is-draining","errorCode":null,"errorMessage":"Unable to start %s because the node is draining.","messagePattern":"Unable to start (.+?) because the node is draining\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/service/StorageService.java","lineNumber":4157,"sourceCode":"\n    /**\n     * Remove a postshutdownhook\n     */\n    public synchronized boolean removePostShutdownHook(Runnable hook)\n    {\n        return postShutdownHooks.remove(hook);\n    }\n\n    /**\n     * Some services are shutdown during draining and we should not attempt to start them again.\n     *\n     * @param service - the name of the service we are trying to start.\n     * @throws IllegalStateException - an exception that nodetool is able to convert into a message to display to the user\n     */\n    synchronized void checkServiceAllowedToStart(String service)\n    {\n        if (isDraining()) // when draining isShutdown is also true, so we check first to return a more accurate message\n            throw new IllegalStateException(String.format(\"Unable to start %s because the node is draining.\", service));\n\n        if (isShutdown()) // do not rely on operationMode in case it gets changed to decommissioned or other\n            throw new IllegalStateException(String.format(\"Unable to start %s because the node was drained.\", service));\n\n        if (!isNormal() && joinRing) // if the node is not joining the ring, it is gossipping-only member which is in STARTING state forever\n            throw new IllegalStateException(String.format(\"Unable to start %s because the node is not in the normal state.\", service));\n    }\n\n    // Never ever do this at home. Used by tests.\n    @VisibleForTesting\n    public IPartitioner setPartitionerUnsafe(IPartitioner newPartitioner)\n    {\n        checkNotNull(newPartitioner, \"newPartitioner is null\");\n        checkState(originalPartitioner == null, \"Already changed the partitioner without resetting\");\n        originalPartitioner = DatabaseDescriptor.setPartitionerUnsafe(newPartitioner);\n        valueFactory = new VersionedValue.VersionedValueFactory(newPartitioner);\n        return originalPartitioner;\n    }","sourceCodeStart":4139,"sourceCodeEnd":4175,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/service/StorageService.java#L4139-L4175","documentation":"checkServiceAllowedToStart(service) refuses to start a service (e.g. thrift/native transport) while the node is draining (nodetool drain). Drain flushes memtables and closes listening connections; starting services mid-drain would reopen the node inconsistently. This is thrown as IllegalStateException so nodetool can display it directly.","triggerScenarios":"Calling startRPCServer()/native transport start (or nodetool enablebinary etc., or service restart) after or during `nodetool drain`.","commonSituations":"Automated restart scripts or service managers (systemd) restarting transport while a drain is in progress or after a drain without restarting the whole JVM.","solutions":["Restart the Cassandra process fully; a drained node cannot resume services in-place","Remove drain steps from automation that expect the node to keep serving","Run `nodetool statusbinary`/logs to confirm the node was drained, then perform a full restart","Reconfigure so transports are only started at boot, not via JMX after drain"],"exampleFix":"// before: restart service via JMX after drain (fails)\nssproxy.startRPCServer();\n// after: full process restart instead of starting services post-drain\nsudo systemctl restart cassandra","handlingStrategy":"try-catch","validationCode":"// check via JMX before starting services\nif (storageService.isDraining()) throw new SkipRestartException(\"node draining\");","typeGuard":null,"tryCatchPattern":"try { storageService.startRPCServer(); } catch (IllegalStateException e) { if (e.getMessage().contains(\"draining\")) { scheduleFullRestart(); } else { throw e; } }","preventionTips":["Never start services via JMX after issuing drain","Drain only when you intend to stop the process","Audit automation for drain + enable* call sequences"],"tags":["drain","node-lifecycle","operational"],"backgroundTag":"invalid-state-transition","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}