{"record":{"id":"f37a0b1a5f30b809","repo":"apache/cassandra","slug":"this-cluster-is-migrating-to-cluster-metadata-can","errorCode":null,"errorMessage":"This cluster is migrating to cluster metadata, can't decommission until that is done.","messagePattern":"This cluster is migrating to cluster metadata, can't decommission until that is done\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/tcm/sequences/SingleNodeSequences.java","lineNumber":69,"sourceCode":"/**\n * This exists simply to group the static entrypoints to sequences that modify a single node\n * e.g. decommission, remove, move\n */\npublic interface SingleNodeSequences\n{\n    Logger logger = LoggerFactory.getLogger(SingleNodeSequences.class);\n\n    /**\n     * Entrypoint to begin node decommission process.\n     *\n     * @param shutdownNetworking if set to true, will also shut down networking on completion\n     * @param force if set to true, will decommission the node even if this would mean there will be not enough nodes\n     *              to satisfy replication factor\n     */\n    static void decommission(boolean shutdownNetworking, boolean force)\n    {\n        if (ClusterMetadataService.instance().isMigrating() || ClusterMetadataService.state() == ClusterMetadataService.State.GOSSIP)\n            throw new IllegalStateException(\"This cluster is migrating to cluster metadata, can't decommission until that is done.\");\n\n        ClusterMetadata metadata = ClusterMetadata.current();\n\n        StorageService.Mode mode = StorageService.instance.operationMode();\n        if (!EnumSet.of(LEAVING, NORMAL, DECOMMISSION_FAILED).contains(mode))\n            throw new UnsupportedOperationException(\"Node in \" + mode + \" state; wait for status to become normal\");\n        logger.debug(\"DECOMMISSIONING\");\n\n        NodeId self = metadata.myNodeId();\n        Collection<Token> tokens = metadata.tokenMap.tokens(self);\n        ReconfigureCMS.maybeReconfigureCMS(metadata, getBroadcastAddressAndPort());\n        MultiStepOperation<?> inProgress = metadata.inProgressSequences.get(self);\n\n        if (inProgress == null)\n        {\n            logger.info(\"starting decommission with {} {}\", metadata.epoch, self);\n            // We reset transferred ranges upon starting a decommission so that we fully stream\n            // anything written since a previous attempt which may not have been persisted to a pending endpoint","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/tcm/sequences/SingleNodeSequences.java#L51-L87","documentation":"SingleNodeSequences.decommission refuses to run while the cluster is still migrating to the TCM (Transactionally Cluster Metadata) subsystem, or while the CMS state machine is still in GOSSIP mode. Decommissioning mid-migration could produce divergent views of cluster membership. The node must wait until migration to cluster metadata is complete.","triggerScenarios":"Calling `nodetool decommission` (which routes to SingleNodeSequences.decommission) while ClusterMetadataService.isMigrating() returns true or ClusterMetadataService.state() == State.GOSSIP.","commonSituations":"Upgrading a cluster from gossip-based metadata to TCM where some nodes have not yet switched; operator runs decommission during a rolling upgrade window; upgrade flags (cassandra.tcm_migration settings) left in a transitional state.","solutions":["Wait for the TCM migration to complete cluster-wide before issuing decommission.","Check migration progress (ClusterMetadataService state / logs showing migration steps) on all nodes.","If migration is stuck, finish or abort the migration per the upgrade procedure, then retry decommission.","Ensure all nodes are upgraded to a TCM-capable version and in agreement on the CMS."],"exampleFix":"// before\nnodetool decommission\n// after\n// verify no migration in progress first\nif (!ClusterMetadataService.instance().isMigrating()\n    && ClusterMetadataService.state() != ClusterMetadataService.State.GOSSIP) {\n    nodetool.decommission();\n}","handlingStrategy":"validation","validationCode":"// guard before decommission\nif (ClusterMetadataService.instance().isMigrating()\n    || ClusterMetadataService.state() == ClusterMetadataService.State.GOSSIP) {\n    throw new RuntimeException(\"TCM migration in progress; postpone decommission\");\n}\nnodetool.decommission();","typeGuard":null,"tryCatchPattern":"try {\n    decommission();\n} catch (IllegalStateException e) {\n    if (e.getMessage().contains(\"migrating to cluster metadata\")) {\n        // retry after migration completes\n        scheduleRetryAfterMigration();\n    } else throw e;\n}","preventionTips":["Track TCM migration status during upgrades and gate topology changes until done.","Do not run decommission/removenode mid-rolling-upgrade.","Monitor ClusterMetadataService state on all nodes before maintenance.","Finish or abort stuck migrations before scheduling decommissions."],"tags":["tcm","migration","decommission","topology"],"backgroundTag":"invalid-state-transition","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}