{"record":{"id":"123538bf19f276da","repo":"apache/cassandra","slug":"unable-to-start-s-because-the-node-was-drained","errorCode":null,"errorMessage":"Unable to start %s because the node was drained.","messagePattern":"Unable to start (.+?) because the node was drained\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/service/StorageService.java","lineNumber":4160,"sourceCode":"     */\n    public synchronized boolean removePostShutdownHook(Runnable hook)\n    {\n        return postShutdownHooks.remove(hook);\n    }\n\n    /**\n     * Some services are shutdown during draining and we should not attempt to start them again.\n     *\n     * @param service - the name of the service we are trying to start.\n     * @throws IllegalStateException - an exception that nodetool is able to convert into a message to display to the user\n     */\n    synchronized void checkServiceAllowedToStart(String service)\n    {\n        if (isDraining()) // when draining isShutdown is also true, so we check first to return a more accurate message\n            throw new IllegalStateException(String.format(\"Unable to start %s because the node is draining.\", service));\n\n        if (isShutdown()) // do not rely on operationMode in case it gets changed to decommissioned or other\n            throw new IllegalStateException(String.format(\"Unable to start %s because the node was drained.\", service));\n\n        if (!isNormal() && joinRing) // if the node is not joining the ring, it is gossipping-only member which is in STARTING state forever\n            throw new IllegalStateException(String.format(\"Unable to start %s because the node is not in the normal state.\", service));\n    }\n\n    // Never ever do this at home. Used by tests.\n    @VisibleForTesting\n    public IPartitioner setPartitionerUnsafe(IPartitioner newPartitioner)\n    {\n        checkNotNull(newPartitioner, \"newPartitioner is null\");\n        checkState(originalPartitioner == null, \"Already changed the partitioner without resetting\");\n        originalPartitioner = DatabaseDescriptor.setPartitionerUnsafe(newPartitioner);\n        valueFactory = new VersionedValue.VersionedValueFactory(newPartitioner);\n        return originalPartitioner;\n    }\n\n    @VisibleForTesting\n    public void resetPartitionerUnsafe()","sourceCodeStart":4142,"sourceCodeEnd":4178,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/service/StorageService.java#L4142-L4178","documentation":"After `nodetool drain` completes, the node is marked SHUTDOWN and checkServiceAllowedToStart(service) blocks any attempt to start services on the drained JVM. A drained node must be restarted as a whole process; services cannot be re-enabled in place. Thrown as IllegalStateException so nodetool renders it to the operator.","triggerScenarios":"Calling startRPCServer()/start native transport (or nodetool enablebinary/enablethrift) on a node whose drain completed (isShutdown() == true).","commonSituations":"Trying to 'un-drain' a node via nodetool; monitoring/auto-healing scripts re-enabling transports after maintenance that ended with drain.","solutions":["Fully restart the Cassandra process (systemctl restart cassandra); drain is not reversible in-place","Avoid drain for routine restarts - use decommission for removal or plain stop/start otherwise","Fix automation that issues enablebinary/enablethrift after drain","Verify with `nodetool statusbinary` and operation mode in logs before attempting to start services"],"exampleFix":"// before: attempt to re-enable transport on drained node\nnodetool enablebinary  # IllegalStateException\n// after: restart the process\nsudo systemctl restart cassandra && nodetool statusbinary","handlingStrategy":"try-catch","validationCode":"ObjectName ss = new ObjectName(\"org.apache.cassandra.db:type=StorageService\");\nboolean shutdown = (boolean) mbs.getAttribute(ss, \"IsShutdown\");\nif (shutdown) performFullRestart();","typeGuard":null,"tryCatchPattern":"try { enableNativeTransport(); } catch (IllegalStateException e) { if (e.getMessage().contains(\"was drained\")) { triggerProcessRestart(); } else { throw e; } }","preventionTips":["Treat drain as one-way until JVM restart","Use plain stop/start for maintenance that must resume","Add health checks that detect SHUTDOWN operation mode"],"tags":["drain","shutdown","node-lifecycle"],"backgroundTag":"invalid-state-transition","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}