apache/cassandra · error · ConfigurationException

Cannot alter RF while some endpoints are not in normal state

Error message

Cannot alter RF while some endpoints are not in normal state (no range movements):

What it means

ALTER KEYSPACE that changes replication factor is rejected when some replica endpoints for the keyspace are not in NORMAL status (e.g. bootstrapping, decommissioning, moving). Changing RF during pending range movements would leave ownership in an inconsistent state, so Cassandra forbids it until all nodes settle.

Source

Thrown at src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java:180

        ClusterMetadata metadata = ClusterMetadata.current();
        NodeId nodeId = metadata.directory.peerId(FBUtilities.getBroadcastAddressAndPort());
        Set<InetAddressAndPort> notNormalEndpoints = metadata.directory.states.entrySet().stream().filter(e -> !e.getKey().equals(nodeId)).filter(e -> {
            switch (e.getValue())
            {
                case BOOTSTRAPPING:
                case LEAVING:
                case MOVING:
                    return true;
                default:
                    return false;
            }

        }).map(e -> metadata.directory.endpoint(e.getKey())).collect(Collectors.toSet());

        if (!notNormalEndpoints.isEmpty())
        {
            throw new ConfigurationException("Cannot alter RF while some endpoints are not in normal state (no range movements): " + notNormalEndpoints);
        }
    }

    private void validateTransientReplication(KeyspaceMetadata current, KeyspaceMetadata proposed)
    {
        //If there is no read traffic there are some extra alterations you can safely make, but this is so atypical
        //that a good default is to not allow unsafe changes
        if (allow_unsafe_transient_changes)
            return;

        ReplicationFactor oldRF = current.replicationStrategy.getReplicationFactor();
        ReplicationFactor newRF = proposed.replicationStrategy.getReplicationFactor();

        int oldTrans = oldRF.transientReplicas();
        int oldFull = oldRF.fullReplicas;
        int newTrans = newRF.transientReplicas();
        int newFull = newRF.fullReplicas;

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Wait until all nodes are in Normal state (`nodetool status`, no UJ/UL/UN nodes), then retry the ALTER KEYSPACE.
  2. Finish or abort the in-flight operation (complete bootstrap, finish decommission/repair) before altering RF.
  3. Alter RF one node/operation at a time, ensuring `nodetool cleanup`/repairs complete between changes.

Example fix

// before (during bootstrap)
session.execute("ALTER KEYSPACE ks WITH replication = {'class':'NetworkTopologyStrategy','dc1':3}");
// after: wait for normal state
assert allEndpointsNormal();
session.execute("ALTER KEYSPACE ks WITH replication = {'class':'NetworkTopologyStrategy','dc1':3}");
Defensive patterns

Strategy: retry

Validate before calling

Set<InetAddress> notNormal = notNormalEndpointsFor(ks); if (!notNormal.isEmpty()) throw new IllegalStateException("defer ALTER KEYSPACE, non-normal: " + notNormal);

Try / catch

try { session.execute(alterKeyspace); } catch (ConfigurationException e) { if (e.getMessage().startsWith("Cannot alter RF while some endpoints")) { waitUntilAllNormal(); retry(); } else throw e; }

Prevention

When it happens

Trigger: `ALTER KEYSPACE ks WITH replication = {...}` changing RF while a node is bootstrapping/leaving/moving ranges for that keyspace (nodetool status shows non-UP/NORMAL or non-UL states).

Common situations: Cluster scaling in progress: someone adds a node (bootstrap) and simultaneously alters RF; also common during decommission or repair-driven range movements in mixed-status clusters.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/de69c9820f6727a5. Report an issue: GitHub.