{"record":{"id":"17458e3df58dcb3a","repo":"apache/cassandra","slug":"cannot-perform-lwt-operation-as-there-is-more-than","errorCode":null,"errorMessage":"Cannot perform LWT operation as there is more than one (%d) pending range movement","messagePattern":"Cannot perform LWT operation as there is more than one \\((.+?)\\) pending range movement","errorType":"exception","errorClass":"UnavailableException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/locator/ReplicaPlans.java","lineNumber":782,"sourceCode":"            liveAndDown = liveAndDown.filter(InOurDc.replicas());\n        }\n\n        ReplicaLayout.ForTokenWrite live = liveAndDown.filter(FailureDetector.isReplicaAlive);\n\n        // TODO: this should use assureSufficientReplicas\n        int participants = liveAndDown.all().size();\n        int requiredParticipants = participants / 2 + 1; // See CASSANDRA-8346, CASSANDRA-833\n\n        if (throwOnInsufficientLiveReplicas)\n        {\n            if (live.all().size() < requiredParticipants)\n                throw UnavailableException.create(consistencyForPaxos, requiredParticipants, live.all().size());\n\n            // We cannot allow CAS operations with 2 or more pending endpoints, see #8346.\n            // Note that we fake an impossible number of required nodes in the unavailable exception\n            // to nail home the point that it's an impossible operation no matter how many nodes are live.\n            if (liveAndDown.pending().size() > 1)\n                throw new UnavailableException(String.format(\"Cannot perform LWT operation as there is more than one (%d) pending range movement\", liveAndDown.all().size()),\n                                               consistencyForPaxos,\n                                               participants + 1,\n                                               live.all().size());\n        }\n\n        return new ReplicaPlan.ForPaxosWrite(keyspace,\n                                             consistencyForPaxos,\n                                             liveAndDown.pending(),\n                                             liveAndDown.all(),\n                                             live.all(),\n                                             live.all(),\n                                             requiredParticipants,\n                                             (newClusterMetadata) -> forPaxos(newClusterMetadata, keyspace, key, consistencyForPaxos, false),\n                                             metadata.epoch);\n    }\n\n    private static <E extends Endpoints<E>> E candidatesForRead(Keyspace keyspace,\n                                                                @Nullable Index.QueryPlan indexQueryPlan,","sourceCodeStart":764,"sourceCodeEnd":800,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/locator/ReplicaPlans.java#L764-L800","documentation":"Lightweight transactions (Paxos-based CAS) are unsafe when more than one range movement is pending in the cluster (CASSANDRA-8346): concurrent pending endpoint transitions can violate CAS linearizability. ReplicaPlans deliberately throws an 'impossible' UnavailableException (requiring participants+1 nodes) so the operation fails fast no matter how many nodes are live.","triggerScenarios":"Performing an INSERT/UPDATE ... IF NOT EXISTS (or other LWT) while the cluster has 2 or more pending endpoints, e.g. during overlapping bootstrap/decommission operations or range movements.","commonSituations":"Running LWTs concurrently with multiple bootstrap or decommission operations; rebalancing token ranges while application traffic uses IF conditions; multi-node repair/move in progress.","solutions":["Wait until pending range movements complete (check `nodetool netstats` / `nodetool status`) and retry the LWT","Serialize topology changes: avoid running more than one bootstrap/decommission at a time when LWTs are in use","Temporarily route LWT traffic away from affected ranges or pause it during topology changes"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// before issuing LWTs, ensure no pending range movements\n// check via JMX: StorageService.operationMode == NORMAL and pending ranges empty\nif (!\"NORMAL\".equals(StorageService.instance.getOperationMode()))\n    throw new IllegalStateException(\"Topology changes in progress; defer LWTs\");","typeGuard":null,"tryCatchPattern":"try { session.execute(lwtStatement); }\ncatch (UnavailableException e) {\n    if (e.getMessage().contains(\"pending range movement\")) { Thread.sleep(backoff); retryWithLimit(); }\n}","preventionTips":["Never run LWT traffic concurrently with multiple bootstrap/decommission operations","Serialize topology changes in clusters that rely on lightweight transactions","Monitor pending endpoints via nodetool and gate LWT traffic accordingly"],"tags":["lwt","paxos","topology-change"],"backgroundTag":"invalid-state-transition","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}