apache/cassandra · error · IllegalArgumentException

Can not decommission a node that has an in-progress sequence

Error message

Can not decommission a node that has an in-progress sequence

What it means

decommission throws IllegalArgumentException when the node's ClusterMetadata contains an in-progress multi-step sequence that is neither a bootstrap nor a leave. An unrelated in-flight sequence means running decommission concurrently would conflict with cluster metadata invariants. The existing sequence must complete or be cancelled first.

Source

Thrown at src/java/org/apache/cassandra/tcm/sequences/SingleNodeSequences.java:103

            logger.info("starting decommission with {} {}", metadata.epoch, self);
            // We reset transferred ranges upon starting a decommission so that we fully stream
            // anything written since a previous attempt which may not have been persisted to a pending endpoint
            SystemKeyspace.resetTransferredRanges();
            logger.info("done resetting transferred ranges {} {}", metadata.epoch, self);
            ClusterMetadataService.instance().commit(new PrepareLeave(self,
                                                                      force,
                                                                      ClusterMetadataService.instance().placementProvider(),
                                                                      LeaveStreams.Kind.UNBOOTSTRAP),
                                                     m -> m,
                                                     failureHandler("PrepareLeave", StorageService.instance::markDecommissionFailed));
        }
        else if (InProgressSequences.isLeave(inProgress))
        {
            logger.info("Resuming decommission @ {} (current epoch = {}): {}", inProgress.latestModification, metadata.epoch, inProgress.status());
        }
        else
        {
            throw new IllegalArgumentException("Can not decommission a node that has an in-progress sequence");
        }

        StorageService.instance.clearTransientMode();
        InProgressSequences.finishInProgressSequences(self);
        Gossiper.instance.unsafeBroadcastLeftStatus(FBUtilities.getBroadcastAddressAndPort(),
                                                    tokens,
                                                    metadata.directory.allJoinedEndpoints());
        if (shutdownNetworking)
            StorageService.instance.shutdownNetworking();
    }

    static void abortDecommission(String nodeId)
    {
        abortHelper(nodeId, MultiStepOperation.Kind.LEAVE, DECOMMISSION_FAILED);
    }

    /**
     * Entrypoint to begin node removal process

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Wait for the in-progress sequence to finish, then decommission.
If the sequence is actually a decommission (leave), it will be resumed automatically instead of throwing.
Cancel or complete the stuck operation via the appropriate repair/abort tooling before retrying.
Inspect ClusterMetadata inProgressSequences (logs/nodetool) to identify which sequence blocks decommission.

Example fix

// before
nodetool decommission  // while a move sequence is in progress
// after
MultiStepOperation<?> inProgress = ClusterMetadata.current().inProgressSequences.get(self);
if (inProgress == null || InProgressSequences.isBootstrap(inProgress) || InProgressSequences.isLeave(inProgress)) {
    nodetool.decommission();
}

Defensive patterns

Strategy: validation

Validate before calling

MultiStepOperation<?> inProgress =
    ClusterMetadata.current().inProgressSequences.get(ClusterMetadata.current().myNodeId());
if (inProgress != null
    && !InProgressSequences.isBootstrap(inProgress)
    && !InProgressSequences.isLeave(inProgress)) {
    throw new RuntimeException("In-progress sequence " + inProgress + " blocks decommission");
}

Try / catch

try {
    decommission();
} catch (IllegalArgumentException e) {
    if (e.getMessage().contains("in-progress sequence")) {
        waitForInProgressSequenceToFinish();
        decommission();
    } else throw e;
}

Prevention

Check for in-progress sequences (bootstrap/leave/replace) before topology changes.
Serialize topology operations; never overlap decommission with other sequences.
Clean up stale sequences after crashes before issuing new operations.
Monitor TCM logs for unfinished multi-step operations.

When it happens

Trigger: Calling decommission while metadata.inProgressSequences for this node holds a sequence other than bootstrap/leave (e.g. a replace, move, or rebuild sequence in progress).

Common situations: Operator decommissions while a prior topology operation (bootstrap, replace, repair-driven sequence) is unfinished; a crashed operation left a stale in-progress sequence registered in TCM.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/d03b79750695afa0. Report an issue: GitHub.