Unrecoverable error while scrubbing %s.Scrubbing cannot cont

Error message

Unrecoverable error while scrubbing %s.Scrubbing cannot continue. The sstable will be marked for deletion. You can attempt manual recovery from the pre-scrub snapshot. You can also run nodetool repair to transfer the data from a healthy replica, if any.

What it means

During a BTI sstable scrub, if an unrecoverable error occurs while reading a partition (scrub cannot resync to the next partition), the scrubber logs this warning and stops scrubbing the sstable. The sstable is then marked for deletion and the operator is pointed at the pre-scrub snapshot or nodetool repair as recovery paths. This is a warning-level output, not an exception: scrub gives up on this sstable rather than crashing the node.

Source

Thrown at src/java/org/apache/cassandra/io/sstable/format/bti/BtiTableScrubber.java:237

                        badPartitions++;
                        if (!seekToNextPartition())
                            break;
                    }
                }
                else
                {
                    throwIfCannotContinue(key, th);

                    badPartitions++;
                    if (indexIterator != null)
                    {
                        outputHandler.warn("Partition starting at position %d is unreadable; skipping to next", dataStart);
                        if (!seekToNextPartition())
                            break;
                    }
                    else
                    {
                        outputHandler.warn("Unrecoverable error while scrubbing %s." +
                                           "Scrubbing cannot continue. The sstable will be marked for deletion. " +
                                           "You can attempt manual recovery from the pre-scrub snapshot. " +
                                           "You can also run nodetool repair to transfer the data from a healthy replica, if any.",
                                           sstable);
                        // There's no way to resync and continue. Give up.
                        break;
                    }
                }
            }
        }
    }


    private boolean indexAvailable()
    {
        return indexIterator != null && !indexIterator.isExhausted();
    }

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Restore the sstable from the pre-scrub snapshot (nodetool listsnapshots; copy from the snapshot dir) and re-attempt or replace the data
Run nodetool repair to stream the missing data from a healthy replica
If replicas hold the data, let the corrupted sstable be marked for deletion and rely on repair/anti-entropy to restore consistency
Check disk health (dmesg, SMART) and replace failing hardware before re-running scrub

Example fix

// before (damaged sstable, no snapshot)
$ nodetool scrub keyspace1 standard1  // scrub aborts, sstable deleted
// after
$ nodetool listsnapshots  # find pre-scrub snapshot
$ nodetool repair keyspace1  // re-stream data from healthy replicas

Defensive patterns

Strategy: fallback

Validate before calling

// before scrub: ensure snapshot exists and replicas are healthy
boolean hasSnapshot = java.nio.file.Files.exists(java.nio.file.Path.of(sstableDir, "snapshots", "pre-scrub"));
boolean replicasHealthy = Keyspace.open(keyspace).getReplicationStrategy()
        .getNaturalReplicasForToken(sstable.first.getToken()).stream()
        .allMatch(ep -> FailureDetector.instance.isAlive(ep));
if (!hasSnapshot || !replicasHealthy) throw new IllegalStateException("Take a snapshot and verify replicas before scrubbing");

Try / catch

// scrub is invoked via nodetool/StorageService; wrap at orchestration level
try {
    Scrubber scrubber = sstable.getScrubber(...);
    scrubber.scrub();
} catch (IOError e) {
    logger.warn("Scrub gave up on {} (unrecoverable). Restore from pre-scrub snapshot or run repair.", sstable, e);
}

Prevention

Always take a snapshot (nodetool snapshot) before running scrub
Keep RF >= 3 so nodetool repair can restore data lost from a corrupt sstable
Monitor disk health (SMART) and filesystem errors; scrub is a symptom of corruption, not a fix for hardware
Run scrub during low traffic and inspect outputHandler warnings for unreadable partitions

When it happens

Trigger: scrubInternal encounters a partition whose data cannot be read or skipped in a way that lets the scrubber seek to the next partition — e.g. corrupted partition index/row data in a BTI sstable such that seekToNextPartition() also fails.

Common situations: Disk corruption or bit rot on an sstable; failed writes leaving a truncated/corrupt BTI data file; hardware issues; running scrub on sstables damaged by an earlier crash or bad disk sector.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/ea4742ce5a98fc20. Report an issue: GitHub.