{"record":{"id":"2abeca5042d3ff7c","repo":"apache/cassandra","slug":"an-error-occurred-while-scrubbing-the-partition-wi","errorCode":null,"errorMessage":"An error occurred while scrubbing the partition with key '%s'.  Skipping corrupt data in counter tables will result in undercounts for the affected counters (see CASSANDRA-2759 for more details), so by default the scrub will stop at this point.  If you would like to skip the row anyway and continue scrubbing, re-run the scrub with the --skip-corrupted option.","messagePattern":"An error occurred while scrubbing the partition with key '(.+?)'\\.  Skipping corrupt data in counter tables will result in undercounts for the affected counters \\(see CASSANDRA-2759 for more details\\), so by default the scrub will stop at this point\\.  If you would like to skip the row anyway and continue scrubbing, re-run the scrub with the --skip-corrupted option\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/io/sstable/format/SortedTableScrubber.java","lineNumber":352,"sourceCode":"\n    private void saveOutOfOrderPartition(DecoratedKey prevKey, DecoratedKey key, UnfilteredRowIterator iterator)\n    {\n        // TODO bitch if the row is too large?  if it is there's not much we can do ...\n        outputHandler.warn(\"Out of order partition detected (%s found after %s)\", keyString(key), keyString(prevKey));\n        outOfOrder.add(ImmutableBTreePartition.create(iterator));\n    }\n\n    protected static void throwIfFatal(Throwable th)\n    {\n        if (th instanceof Error && !(th instanceof AssertionError || th instanceof IOError))\n            throw (Error) th;\n    }\n\n    protected void throwIfCannotContinue(DecoratedKey key, Throwable th)\n    {\n        if (isCommutative && !options.skipCorrupted)\n        {\n            outputHandler.warn(\"An error occurred while scrubbing the partition with key '%s'.  Skipping corrupt \" +\n                               \"data in counter tables will result in undercounts for the affected \" +\n                               \"counters (see CASSANDRA-2759 for more details), so by default the scrub will \" +\n                               \"stop at this point.  If you would like to skip the row anyway and continue \" +\n                               \"scrubbing, re-run the scrub with the --skip-corrupted option.\",\n                               keyString(key));\n            throw new IOError(th);\n        }\n    }\n\n\n    public static class ScrubInfo extends CompactionInfo.Holder\n    {\n        private final RandomAccessReader dataFile;\n        private final SSTableReader sstable;\n        private final TimeUUID scrubCompactionId;\n        private final Lock fileReadLock;\n\n        public ScrubInfo(RandomAccessReader dataFile, SSTableReader sstable, Lock fileReadLock)","sourceCodeStart":334,"sourceCodeEnd":370,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/sstable/format/SortedTableScrubber.java#L334-L370","documentation":"nodetool scrub encountered an unreadable/corrupt partition on a counter table. Because skipping corrupt data in counter tables permanently undercounts affected counters (CASSANDRA-2759), the scrubber warns and aborts (throws IOError) unless --skip-corrupted was passed. This is a deliberate safety stop to prevent silent counter data loss.","triggerScenarios":"Running nodetool scrub on a counter table containing a corrupted partition (torn write, disk corruption, bad SSTable) without --skip-corrupted.","commonSituations":"Recovering from disk failures or crashed hardware; scrubbing after unclean shutdowns on counter-heavy tables; repair of corrupt SSTables found by nodetool verify.","solutions":["First back up the affected SSTables, then re-run scrub with --skip-corrupted if you accept undercounts for the affected counters.","Attempt to restore affected counters: re-run counter increments or rebuild the table from application data/other replicas.","Check replicas: if other replicas have healthy data, run repair (nodetool repair) to restore the partition instead of skipping.","Investigate root cause of corruption (disk health, filesystem, CASSANDRA-version bugs) to prevent recurrence."],"exampleFix":"// before: scrub aborts on counter table corruption\nnodetool scrub <keyspace> <counter_table>\n// after: accept undercounts and continue (backup SSTables first!)\nnodetool scrub --skip-corrupted <keyspace> <counter_table>\nnodetool repair <keyspace> <counter_table>","handlingStrategy":"fallback","validationCode":"# Detect corruption before scrub so you can plan (backup first!)\nnodetool verify <keyspace> <counter_table>   # reports corrupt SSTables non-destructively\n# Snapshot before any scrub:\nnodetool snapshot <keyspace> --table <counter_table>","typeGuard":null,"tryCatchPattern":"// CLI-level: scrub aborts with IOError; handle by planning recovery\n// nodetool scrub ks counter_tbl  -> aborts on counter corruption\n// Fallback sequence: snapshot -> scrub --skip-corrupted -> repair -> validate counters from app data","preventionTips":["Run nodetool verify periodically to catch corruption before scrub operations.","Always snapshot counter tables before scrubbing.","Repair counter tables promptly after unclean shutdowns or disk issues.","Monitor disk health (SMART) to reduce torn-write corruption risk.","Design counters with compensating application-side sources of truth where exact counts matter."],"tags":["cassandra","scrub","counter-table","data-corruption"],"backgroundTag":"checksum-mismatch","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}