apache/cassandra · error

Key in sstable not owned by local ranges

Error message

Key %s in sstable %s not owned by local ranges %s

What it means

During nodetool verify, verifySSTable validates every key in the sstable's index against the token ranges this node owns (rangeOwnHelper.validate). A key outside local ranges is warned (with the key, sstable, and owned ranges), recorded, and — per verify's fail-fast policy — marked and the original Throwable rethrown at the end, making verification fail. It detects misplaced sstable data, often after topology changes or misrestored files.

Solutions

  1. Remove or relocate the misplaced sstable (nodetool decommission/cleanup semantics): run nodetool cleanup to drop keys not owned locally
  2. If the sstable was restored from another node, restore from a snapshot belonging to this node instead
  3. Check cassandra.yaml token configuration (initial_token, num_tokens) matches the ring; run a full ring repair after topology changes
  4. Re-run nodetool verify after cleanup to confirm the ring is consistent

Example fix

// operator fix, not code
nodetool cleanup   # drop keys this node no longer owns
nodetool verify    # re-check
Defensive patterns

Strategy: validation

Validate before calling

// Before verify, confirm node ownership covers sstable tokens:
for (DecoratedKey k : sampledKeys) {
    if (!StorageService.instance.getLocalRanges(keyspace).stream()
            .anyMatch(r -> r.contains(k.getToken())))
        throw new IllegalStateException("Key " + k + " not owned locally; run nodetool cleanup");
}

Try / catch

try { verifier.verify(sstable); } catch (Throwable t) { logger.error("verify failed: sstable holds out-of-range keys", t); scheduleCleanup(); }

Prevention

When it happens

Trigger: Running nodetool verify (with extended verification) on a node where an sstable contains a key whose token is not in the node's current local ranges; validate() throws and markAndThrow(t) schedules the failure.

Common situations: Sstables copied/restored from another node or datacenter; token/ring changes after bootstrapping, decommissioning, or a wrong initial_token/num_tokens config; running verify on a node whose ownership shifted mid-operation.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/26d85479f0cebbdf. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/io/sstable/format/SortedTableVerifier.java:311

                DecoratedKey key = null;
                try
                {
                    key = sstable.decorateKey(ByteBufferUtil.readWithShortLength(dataFile));
                }
                catch (Throwable th)
                {
                    markAndThrow(th);
                }

                if (options.checkOwnsTokens && ownedRanges.size() > 0 && !(cfs.getPartitioner() instanceof LocalPartitioner))
                {
                    try
                    {
                        rangeOwnHelper.validate(key);
                    }
                    catch (Throwable t)
                    {
                        outputHandler.warn(t, "Key %s in sstable %s not owned by local ranges %s", key, sstable, ownedRanges);
                        markAndThrow(t);
                    }
                }

                ByteBuffer currentIndexKey = indexIterator.key();
                long nextRowPositionFromIndex = 0;
                try
                {
                    nextRowPositionFromIndex = indexIterator.advance()
                                               ? indexIterator.dataPosition()
                                               : dataFile.length();
                }
                catch (Throwable th)
                {
                    markAndThrow(th);
                }

                long dataStart = dataFile.getFilePointer();

View on GitHub (pinned to 88fd0f6a0e)