apache/cassandra · error · UnavailableException

Cannot perform LWT operation as there is more than one (%d)

Error message

Cannot perform LWT operation as there is more than one (%d) pending range movement

What it means

Lightweight transactions (Paxos-based CAS) are unsafe when more than one range movement is pending in the cluster (CASSANDRA-8346): concurrent pending endpoint transitions can violate CAS linearizability. ReplicaPlans deliberately throws an 'impossible' UnavailableException (requiring participants+1 nodes) so the operation fails fast no matter how many nodes are live.

Source

Thrown at src/java/org/apache/cassandra/locator/ReplicaPlans.java:782

            liveAndDown = liveAndDown.filter(InOurDc.replicas());
        }

        ReplicaLayout.ForTokenWrite live = liveAndDown.filter(FailureDetector.isReplicaAlive);

        // TODO: this should use assureSufficientReplicas
        int participants = liveAndDown.all().size();
        int requiredParticipants = participants / 2 + 1; // See CASSANDRA-8346, CASSANDRA-833

        if (throwOnInsufficientLiveReplicas)
        {
            if (live.all().size() < requiredParticipants)
                throw UnavailableException.create(consistencyForPaxos, requiredParticipants, live.all().size());

            // We cannot allow CAS operations with 2 or more pending endpoints, see #8346.
            // Note that we fake an impossible number of required nodes in the unavailable exception
            // to nail home the point that it's an impossible operation no matter how many nodes are live.
            if (liveAndDown.pending().size() > 1)
                throw new UnavailableException(String.format("Cannot perform LWT operation as there is more than one (%d) pending range movement", liveAndDown.all().size()),
                                               consistencyForPaxos,
                                               participants + 1,
                                               live.all().size());
        }

        return new ReplicaPlan.ForPaxosWrite(keyspace,
                                             consistencyForPaxos,
                                             liveAndDown.pending(),
                                             liveAndDown.all(),
                                             live.all(),
                                             live.all(),
                                             requiredParticipants,
                                             (newClusterMetadata) -> forPaxos(newClusterMetadata, keyspace, key, consistencyForPaxos, false),
                                             metadata.epoch);
    }

    private static <E extends Endpoints<E>> E candidatesForRead(Keyspace keyspace,
                                                                @Nullable Index.QueryPlan indexQueryPlan,

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Wait until pending range movements complete (check `nodetool netstats` / `nodetool status`) and retry the LWT
Serialize topology changes: avoid running more than one bootstrap/decommission at a time when LWTs are in use
Temporarily route LWT traffic away from affected ranges or pause it during topology changes

Defensive patterns

Strategy: retry

Validate before calling

// before issuing LWTs, ensure no pending range movements
// check via JMX: StorageService.operationMode == NORMAL and pending ranges empty
if (!"NORMAL".equals(StorageService.instance.getOperationMode()))
    throw new IllegalStateException("Topology changes in progress; defer LWTs");

Try / catch

try { session.execute(lwtStatement); }
catch (UnavailableException e) {
    if (e.getMessage().contains("pending range movement")) { Thread.sleep(backoff); retryWithLimit(); }
}

Prevention

Never run LWT traffic concurrently with multiple bootstrap/decommission operations
Serialize topology changes in clusters that rely on lightweight transactions
Monitor pending endpoints via nodetool and gate LWT traffic accordingly

When it happens

Trigger: Performing an INSERT/UPDATE ... IF NOT EXISTS (or other LWT) while the cluster has 2 or more pending endpoints, e.g. during overlapping bootstrap/decommission operations or range movements.

Common situations: Running LWTs concurrently with multiple bootstrap or decommission operations; rebalancing token ranges while application traffic uses IF conditions; multi-node repair/move in progress.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/17458e3df58dcb3a. Report an issue: GitHub.