{"record":{"id":"26d85479f0cebbdf","repo":"apache/cassandra","slug":"key-s-in-sstable-s-not-owned-by-local-ranges-s","errorCode":null,"errorMessage":"Key %s in sstable %s not owned by local ranges %s","messagePattern":"Key (.+?) in sstable (.+?) not owned by local ranges (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/io/sstable/format/SortedTableVerifier.java","lineNumber":311,"sourceCode":"                DecoratedKey key = null;\n                try\n                {\n                    key = sstable.decorateKey(ByteBufferUtil.readWithShortLength(dataFile));\n                }\n                catch (Throwable th)\n                {\n                    markAndThrow(th);\n                }\n\n                if (options.checkOwnsTokens && ownedRanges.size() > 0 && !(cfs.getPartitioner() instanceof LocalPartitioner))\n                {\n                    try\n                    {\n                        rangeOwnHelper.validate(key);\n                    }\n                    catch (Throwable t)\n                    {\n                        outputHandler.warn(t, \"Key %s in sstable %s not owned by local ranges %s\", key, sstable, ownedRanges);\n                        markAndThrow(t);\n                    }\n                }\n\n                ByteBuffer currentIndexKey = indexIterator.key();\n                long nextRowPositionFromIndex = 0;\n                try\n                {\n                    nextRowPositionFromIndex = indexIterator.advance()\n                                               ? indexIterator.dataPosition()\n                                               : dataFile.length();\n                }\n                catch (Throwable th)\n                {\n                    markAndThrow(th);\n                }\n\n                long dataStart = dataFile.getFilePointer();","sourceCodeStart":293,"sourceCodeEnd":329,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/sstable/format/SortedTableVerifier.java#L293-L329","documentation":"During nodetool verify, verifySSTable validates every key in the sstable's index against the token ranges this node owns (rangeOwnHelper.validate). A key outside local ranges is warned (with the key, sstable, and owned ranges), recorded, and — per verify's fail-fast policy — marked and the original Throwable rethrown at the end, making verification fail. It detects misplaced sstable data, often after topology changes or misrestored files.","triggerScenarios":"Running nodetool verify (with extended verification) on a node where an sstable contains a key whose token is not in the node's current local ranges; validate() throws and markAndThrow(t) schedules the failure.","commonSituations":"Sstables copied/restored from another node or datacenter; token/ring changes after bootstrapping, decommissioning, or a wrong initial_token/num_tokens config; running verify on a node whose ownership shifted mid-operation.","solutions":["Remove or relocate the misplaced sstable (nodetool decommission/cleanup semantics): run nodetool cleanup to drop keys not owned locally","If the sstable was restored from another node, restore from a snapshot belonging to this node instead","Check cassandra.yaml token configuration (initial_token, num_tokens) matches the ring; run a full ring repair after topology changes","Re-run nodetool verify after cleanup to confirm the ring is consistent"],"exampleFix":"// operator fix, not code\nnodetool cleanup   # drop keys this node no longer owns\nnodetool verify    # re-check","handlingStrategy":"validation","validationCode":"// Before verify, confirm node ownership covers sstable tokens:\nfor (DecoratedKey k : sampledKeys) {\n    if (!StorageService.instance.getLocalRanges(keyspace).stream()\n            .anyMatch(r -> r.contains(k.getToken())))\n        throw new IllegalStateException(\"Key \" + k + \" not owned locally; run nodetool cleanup\");\n}","typeGuard":null,"tryCatchPattern":"try { verifier.verify(sstable); } catch (Throwable t) { logger.error(\"verify failed: sstable holds out-of-range keys\", t); scheduleCleanup(); }","preventionTips":["Run nodetool cleanup after any topology change (bootstrap/decommission/token change)","Never copy sstables between nodes with different token ownership","Keep initial_token/num_tokens configuration consistent and documented","Run verify after restores and ring changes"],"tags":["verify","token-ranges","topology","sstable"],"backgroundTag":"internal-invariant-violation","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}