{"record":{"id":"e03c27fcb7daeb87","repo":"apache/cassandra","slug":"read-d-live-rows-and-d-tombstone-cells-for-query","errorCode":null,"errorMessage":"Read %d live rows and %d tombstone cells for query %1.512s; token %s (see tombstone_warn_threshold)","messagePattern":"Read (.+?) live rows and (.+?) tombstone cells for query %1\\.512s; token (.+?) \\(see tombstone_warn_threshold\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/db/ReadCommand.java","lineNumber":739,"sourceCode":"            @Override\n            public void onClose()\n            {\n                recordLatency(metric, nanoTime() - startTimeNanos);\n\n                metric.tombstoneScannedHistogram.update(tombstones);\n                metric.liveScannedHistogram.update(liveRows);\n                metric.totalRowsRead.inc(liveRows);\n\n                boolean warnTombstones = tombstones > warningThreshold && respectTombstoneThresholds;\n                if (warnTombstones)\n                {\n                    String msg = String.format(\n                            \"Read %d live rows and %d tombstone cells for query %1.512s; token %s (see tombstone_warn_threshold)\",\n                            liveRows, tombstones, ReadCommand.this.toCQLString(), currentKey.getToken());\n                    if (trackWarnings)\n                        MessageParams.add(ParamType.TOMBSTONE_WARNING, tombstones);\n                    else\n                        ClientWarn.instance.warn(msg);\n                    if (tombstones < failureThreshold)\n                    {\n                        metric.tombstoneWarnings.inc();\n                    }\n\n                    logger.warn(msg);\n                }\n\n                Tracing.trace(\"Read {} live rows and {} tombstone cells{}\",\n                        liveRows, tombstones,\n                        (warnTombstones ? \" (see tombstone_warn_threshold)\" : \"\"));\n            }\n        }\n\n        return Transformation.apply(iter, new MetricRecording());\n    }\n\n    private boolean shouldTrackSize(DataStorageSpec.LongBytesBound warnThresholdBytes, DataStorageSpec.LongBytesBound abortThresholdBytes)","sourceCodeStart":721,"sourceCodeEnd":757,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/db/ReadCommand.java#L721-L757","documentation":"When a read command completes, Cassandra counts tombstone cells; if the count exceeds tombstone_warn_threshold it logs and warns the client that the query read many tombstones, including the CQL text and scanned token. Exceeding tombstone_failure_threshold escalates to a TombstoneOverwhelmingException; this warning is the earlier, non-fatal signal of expensive tombstone scans.","triggerScenarios":"Any SELECT whose coordinator processes more live/tombstone cells than tombstone_warn_threshold (default 1000) during local execution onClose; the warning is sent via ClientWarn when warning tracking is off.","commonSituations":"Wide-partition deletes leaving many tombstones; queries scanning dead cells from expired TTL data; non-sstable-scanner queries over heavily mutated partitions; compacted-pending data.","solutions":["Rewrite the query to be more selective (narrow partition/clustering ranges) so fewer tombstones are scanned.","Run compaction (or alter compaction strategy, e.g. DTCS/TWCS for TTL data) to purge accumulated tombstones.","Raise tombstone_warn_threshold in cassandra.yaml only after confirming the read cost is acceptable.","Check for application patterns issuing repeated range deletes/overwrites and reduce them."],"exampleFix":"// before\nSELECT * FROM ks.events; // scans whole partitions incl. tombstones\n// after\nSELECT * FROM ks.events WHERE day = '2026-09-10' AND bucket = 3;","handlingStrategy":"validation","validationCode":"// keep expected scan cost under tombstone_warn_threshold\nlong approxCells = estimatedPartitionCells(partitionKey); // from your data model\nif (approxCells > 1000) throw new IllegalArgumentException(\"query would scan too many tombstones; narrow it\");","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Model data to avoid dense delete/overwrite patterns that leave tombstones.","Use TWCS/DTCS for TTL-heavy tables and keep gc_grace compaction running.","Monitor the TableMetrics tombstoneWarnings metric and tune queries, not just thresholds."],"tags":["tombstones","read-performance","client-warning","compaction"],"backgroundTag":"tombstone-warning-threshold-exceeded","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}