{"record":{"id":"efb4c2316d33ead1","repo":"apache/cassandra","slug":"tombstoneoverwhelmingexception","errorCode":null,"errorMessage":"TombstoneOverwhelmingException","messagePattern":"TombstoneOverwhelmingException","errorType":"exception","errorClass":"TombstoneOverwhelmingException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/db/ReadCommand.java","lineNumber":701,"sourceCode":"            {\n                countTombstone(marker.clustering());\n                return marker;\n            }\n\n            private void countTombstone(ClusteringPrefix<?> clustering)\n            {\n                ++tombstones;\n                if (tombstones > failureThreshold && respectTombstoneThresholds)\n                {\n                    String query = ReadCommand.this.toCQLString();\n                    Tracing.trace(\"Scanned over {} tombstones for query {}; query aborted (see tombstone_failure_threshold)\", failureThreshold, query);\n                    metric.tombstoneFailures.inc();\n                    if (trackWarnings)\n                    {\n                        MessageParams.remove(ParamType.TOMBSTONE_WARNING);\n                        MessageParams.add(ParamType.TOMBSTONE_FAIL, tombstones);\n                    }\n                    throw new TombstoneOverwhelmingException(tombstones, query, ReadCommand.this.metadata(), currentKey, clustering);\n                }\n            }\n\n            @Override\n            protected void onPartitionClose()\n            {\n                int lr = liveRows - lastReportedLiveRows;\n                int ts = tombstones - lastReportedTombstones;\n\n                if (lr > 0)\n                    metric.topReadPartitionRowCount.addSample(currentKey.getKey(), lr);\n\n                if (ts > 0)\n                    metric.topReadPartitionTombstoneCount.addSample(currentKey.getKey(), ts);\n\n                lastReportedLiveRows = liveRows;\n                lastReportedTombstones = tombstones;\n            }","sourceCodeStart":683,"sourceCodeEnd":719,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/db/ReadCommand.java#L683-L719","documentation":"ReadCommand's tombstone-counting row/merkle iterator throws TombstoneOverwhelmingException when a single query touches more tombstones than tombstone_failure_threshold (default 100k). This protects the node from read explosions caused by mass deletes/expiring data; a client query is aborted rather than allowed to grind the coordinator down.","triggerScenarios":"SELECT (or secondary-index/compaction-adjacent reads) over partitions containing more than tombstone_fail_threshold tombstones — e.g. queries against tables with heavy deletes, frequent TTL expiry of non-local-expiry data, or very old un-compacted SSTables.","commonSituations":"Time-series tables where old partitions are deleted but never fully compacted; queries without partition keys (full-table scans) hitting tombstone-rich SSTables; tombstone_gc delays plus repair-lag leaving resurrection risk un-compacted.","solutions":["Run `nodetool compact` on the affected table to purge tombstoned data, or wait for compaction.","Raise tombstone_failure_threshold / tombstone_warn_threshold temporarily in cassandra.yaml if reads are legitimately tombstone-heavy.","Redesign schema: avoid unbounded deletes, delete by partition, use shorter gc_grace_seconds deliberately, prefer TTL over explicit deletes.","Add LIMIT / narrower partition-key predicates to the query so fewer tombstones are scanned."],"exampleFix":"// before\nsession.execute(\"SELECT * FROM events\"); // scans millions of tombstones\n// after\nsession.execute(\"SELECT * FROM events WHERE day = ? AND bucket = ? LIMIT 1000\", day, bucket); // partition-scoped","handlingStrategy":"try-catch","validationCode":"// Pre-check table health: watch `nodetool tablestats` tombstone counters; alert when\ntombstones_scanned per read approaches tombstone_warn_threshold.","typeGuard":null,"tryCatchPattern":"try { rs = session.execute(query); } catch (DriverException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"tombstones\")) { runCompactOrNarrowQuery(); } else throw e;\n}","preventionTips":["Always scope reads by partition key; avoid full-table scans on delete-heavy tables.","Prefer TTL-based expiry over mass DELETE statements.","Schedule compaction and monitor SSTable tombstone density.","Review gc_grace_seconds and repair cadence."],"tags":["tombstone","read-path","performance","query-aborted"],"backgroundTag":"database-query-failed","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}