Could not reconfigure CMS, operator should run `nodetool…

Error message

Could not reconfigure CMS, operator should run `nodetool cms reconfigure` to make sure CMS placement is correct

What it means

ClusterMetadataService.ensureCMSPlacement catches a failure while trying to automatically reconfigure the CMS replication and logs this warning instead of failing. It tells the operator that automatic CMS placement repair did not succeed and manual reconfiguration via `nodetool cms reconfigure` is needed.

Solutions

Run `nodetool cms reconfigure <rf>` manually once the cluster is healthy.
Check cluster size and node availability against the required CMS replication factor.
Inspect the logged cause (the attached Throwable) for the underlying commit failure.
Fix network/partition issues, then retry startup or run reconfigure again.

Example fix

// before
bin/nodetool status  # confirm nodes up
// after
bin/nodetool cms reconfigure --replication-factor 3

Defensive patterns

Strategy: fallback

Validate before calling

// check cluster size vs required CMS RF before relying on auto-placement
if (liveNodes.size() < requiredCmsRf) logger.warn("too few live nodes for CMS placement");

Try / catch

try { ensureCMSPlacement(); } catch (Throwable t) { runNodetoolCmsReconfigure(); }

Prevention

Start clusters with enough nodes for the CMS replication factor
Keep CMS members healthy and quorum-reachable
Watch startup logs for this warning and reconfigure manually

When it happens

Trigger: A node starts (or topology changes) and ensureCMSPlacement attempts reconfigureCMS(ReplicationParams.meta(metadata)) but the commit fails — e.g. not enough nodes to satisfy placement, unreachable CMS members, or a Paxos/commit timeout.

Common situations: Bootstrapping a cluster too small for the requested CMS replication factor; nodes down or unreachable during startup; multi-DC setups where the CMS placement cannot be satisfied.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/b8bee3d62cd51faa. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/tcm/ClusterMetadataService.java:531

    }

    public void ensureCMSPlacement(ClusterMetadata metadata)
    {
        if (TCM_SKIP_CMS_RECONFIGURATION_AFTER_TOPOLOGY_CHANGE.getBoolean())
        {
            logger.info("Not performing CMS reconfiguration as {} property is set. This should only be used for testing.",
                        TCM_SKIP_CMS_RECONFIGURATION_AFTER_TOPOLOGY_CHANGE.getKey());
            return;
        }

        try
        {
            reconfigureCMS(ReplicationParams.meta(metadata));
        }
        catch (Throwable t)
        {
            JVMStabilityInspector.inspectThrowable(t);
            logger.warn("Could not reconfigure CMS, operator should run `nodetool cms reconfigure` to make sure CMS placement is correct", t);
        }
    }

    public boolean applyFromGossip(ClusterMetadata expected, ClusterMetadata updated)
    {
        logger.debug("Applying from gossip, current={} new={}", expected, updated);
        if (!expected.epoch.isBefore(Epoch.EMPTY))
            throw new IllegalStateException("Can't apply a ClusterMetadata from gossip with epoch " + expected.epoch);
        if (state() != GOSSIP)
            throw new IllegalStateException("Can't apply a ClusterMetadata from gossip when CMSState is not GOSSIP: " + state());

        return log.unsafeSetCommittedFromGossip(expected, updated);
    }

    public void setFromGossip(ClusterMetadata fromGossip)
    {
        logger.debug("Setting from gossip, new={}", fromGossip);
        if (state() != GOSSIP)

View on GitHub (pinned to 88fd0f6a0e)