apache/cassandra · warning

Cannot drain node (did it already happen?)

Error message

Cannot drain node (did it already happen?)

What it means

Drain was requested but the mutation executors are already terminated, meaning the node was already drained (or shut down). The method just logs a warning and returns - the 'error' is benign and defensive against double-drain.

Solutions

1. No action needed - the node is already drained
2. If scripted, check executor state or tolerate the warning and exit 0
3. Restart Cassandra to leave the drained state before further operations

Example fix

// before
nodetool drain
nodetool drain  # second call warns
// after
nodetool drain && echo already-or-just-drained

catch (RuntimeException e) { if (e.getMessage().contains("Cannot drain node")) { /* already drained - treat as success */ } }

When it happens

Trigger: Calling nodetool 'drain' twice; calling drain after 'nodetool stop'/JVM shutdown already drained; a shutdown hook running drain while a user also invoked drain.

Common situations: Automation scripts that retry drain after a timeout; operators running drain manually and then stopping the service, which drains again during shutdown.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• HintsService has already been shut down
• SSTables import has been aborted
• Unable to start because the node was drained.
• Can not fetch log entries during shutdown
• Can not replay during shutdown

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/394b130097f2263a. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/service/StorageService.java:3888

    public String getDrainProgress()
    {
        return String.format("Drained %s/%s ColumnFamilies", remainingCFs, totalCFs);
    }

    /**
     * Shuts node off to writes, empties memtables and the commit log.
     */
    public synchronized void drain() throws IOException, InterruptedException, ExecutionException
    {
        drain(false);
    }

    protected synchronized void drain(boolean isFinalShutdown) throws IOException, InterruptedException, ExecutionException
    {
        if (Stage.areMutationExecutorsTerminated())
        {
            if (!isFinalShutdown)
                logger.warn("Cannot drain node (did it already happen?)");
            return;
        }

        assert !isShutdown;
        isShutdown = true;

        Throwable preShutdownHookThrowable = Throwables.perform(null, preShutdownHooks.stream().map(h -> h::run));
        if (preShutdownHookThrowable != null)
            logger.error("Attempting to continue draining after pre-shutdown hooks returned exception", preShutdownHookThrowable);

        try
        {
            String msg = "starting drain process";
            if (!isFinalShutdown)
                logger.info(msg);
            else
                logger.debug(msg);
            transientMode = Optional.of(Mode.DRAINING);

View on GitHub (pinned to 88fd0f6a0e)