apache/cassandra · error · IllegalStateException

This cluster is migrating to cluster metadata, can't decommi

Error message

This cluster is migrating to cluster metadata, can't decommission until that is done.

What it means

SingleNodeSequences.decommission refuses to run while the cluster is still migrating to the TCM (Transactionally Cluster Metadata) subsystem, or while the CMS state machine is still in GOSSIP mode. Decommissioning mid-migration could produce divergent views of cluster membership. The node must wait until migration to cluster metadata is complete.

Source

Thrown at src/java/org/apache/cassandra/tcm/sequences/SingleNodeSequences.java:69

/**
 * This exists simply to group the static entrypoints to sequences that modify a single node
 * e.g. decommission, remove, move
 */
public interface SingleNodeSequences
{
    Logger logger = LoggerFactory.getLogger(SingleNodeSequences.class);

    /**
     * Entrypoint to begin node decommission process.
     *
     * @param shutdownNetworking if set to true, will also shut down networking on completion
     * @param force if set to true, will decommission the node even if this would mean there will be not enough nodes
     *              to satisfy replication factor
     */
    static void decommission(boolean shutdownNetworking, boolean force)
    {
        if (ClusterMetadataService.instance().isMigrating() || ClusterMetadataService.state() == ClusterMetadataService.State.GOSSIP)
            throw new IllegalStateException("This cluster is migrating to cluster metadata, can't decommission until that is done.");

        ClusterMetadata metadata = ClusterMetadata.current();

        StorageService.Mode mode = StorageService.instance.operationMode();
        if (!EnumSet.of(LEAVING, NORMAL, DECOMMISSION_FAILED).contains(mode))
            throw new UnsupportedOperationException("Node in " + mode + " state; wait for status to become normal");
        logger.debug("DECOMMISSIONING");

        NodeId self = metadata.myNodeId();
        Collection<Token> tokens = metadata.tokenMap.tokens(self);
        ReconfigureCMS.maybeReconfigureCMS(metadata, getBroadcastAddressAndPort());
        MultiStepOperation<?> inProgress = metadata.inProgressSequences.get(self);

        if (inProgress == null)
        {
            logger.info("starting decommission with {} {}", metadata.epoch, self);
            // We reset transferred ranges upon starting a decommission so that we fully stream
            // anything written since a previous attempt which may not have been persisted to a pending endpoint

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Wait for the TCM migration to complete cluster-wide before issuing decommission.
2. Check migration progress (ClusterMetadataService state / logs showing migration steps) on all nodes.
3. If migration is stuck, finish or abort the migration per the upgrade procedure, then retry decommission.
4. Ensure all nodes are upgraded to a TCM-capable version and in agreement on the CMS.

Example fix

// before
nodetool decommission
// after
// verify no migration in progress first
if (!ClusterMetadataService.instance().isMigrating()
    && ClusterMetadataService.state() != ClusterMetadataService.State.GOSSIP) {
    nodetool.decommission();
}

// guard before decommission
if (ClusterMetadataService.instance().isMigrating()
    || ClusterMetadataService.state() == ClusterMetadataService.State.GOSSIP) {
    throw new RuntimeException("TCM migration in progress; postpone decommission");
}
nodetool.decommission();

try {
    decommission();
} catch (IllegalStateException e) {
    if (e.getMessage().contains("migrating to cluster metadata")) {
        // retry after migration completes
        scheduleRetryAfterMigration();
    } else throw e;
}

When it happens

Trigger: Calling `nodetool decommission` (which routes to SingleNodeSequences.decommission) while ClusterMetadataService.isMigrating() returns true or ClusterMetadataService.state() == State.GOSSIP.

Common situations: Upgrading a cluster from gossip-based metadata to TCM where some nodes have not yet switched; operator runs decommission during a rolling upgrade window; upgrade flags (cassandra.tcm_migration settings) left in a transitional state.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Node in state; wait for status to become normal
• Can not decommission a node that has an in-progress sequence
• This cluster is migrating to cluster metadata, can't move un
• Error while decommissioning node:
• Locations differ: {} != {}

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/f37a0b1a5f30b809. Report an issue: GitHub.