Retry failed too. Skipping to next partition (retry's stackt

Error message

Retry failed too. Skipping to next partition (retry's stacktrace follows)

What it means

After a partition read error, the BTI scrubber repositions the data file using the index and retries the partition once. If that retry also fails, this warning logs the retry's stacktrace, the partition is counted as bad, and the scrubber skips to the next partition (via seekToNextPartition) or aborts if no further partition can be located.

Source

Thrown at src/java/org/apache/cassandra/io/sstable/format/bti/BtiTableScrubber.java:218

                    long rowStartFromIndex = dataStartFromIndex + TypeSizes.SHORT_SIZE + currentIndexKey.remaining();
                    outputHandler.output("Retrying from partition index; data is %s bytes starting at %s",
                                         dataSizeFromIndex, rowStartFromIndex);
                    key = sstable.decorateKey(currentIndexKey);
                    try
                    {
                        if (!isIndex)
                            partitionKeyType.validate(key.getKey());
                        dataFile.seek(rowStartFromIndex);

                        if (tryAppend(prevKey, key, writer))
                            prevKey = key;
                    }
                    catch (Throwable th2)
                    {
                        throwIfFatal(th2);
                        throwIfCannotContinue(key, th2);

                        outputHandler.warn(th2, "Retry failed too. Skipping to next partition (retry's stacktrace follows)");
                        badPartitions++;
                        if (!seekToNextPartition())
                            break;
                    }
                }
                else
                {
                    throwIfCannotContinue(key, th);

                    badPartitions++;
                    if (indexIterator != null)
                    {
                        outputHandler.warn("Partition starting at position %d is unreadable; skipping to next", dataStart);
                        if (!seekToNextPartition())
                            break;
                    }
                    else
                    {

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Accept the skip if the data is replicated elsewhere; run `nodetool repair` to re-replicate the lost partitions.
Inspect the logged stacktrace for the root cause (IO error vs malformed data).
If badPartitions is nonzero and data is unrecoverable locally, restore the affected sstables from a snapshot.
Fix the underlying storage issue (disk errors, permissions) before scrubbing again.

Example fix

// Recover data from replicas:
// before: scrub skips partition, local data lost
// after:
$ nodetool repair <keyspace> <table>
// verify with:
$ nodetool verify <keyspace> <table>

Defensive patterns

Strategy: retry

Validate before calling

// Verify replicas hold the data before accepting skipped partitions
$ nodetool repair --dry-run keyspace table

Try / catch

// Count bad partitions post-scrub and trigger repair if any
int badPartitions = scrubResult.getBadPartitions();
if (badPartitions > 0) { /* run nodetool repair */ }

Prevention

Maintain replication factor >= 3 so skipped partitions are recoverable from replicas.
Run nodetool repair after any scrub that reports bad partitions.
Replace failing hardware promptly; repeated retry failures indicate real disk damage.
Avoid filling disks > 80% capacity, which leads to truncated writes.

When it happens

Trigger: `nodetool scrub` on a BTI sstable where both the original read and the index-guided re-seek+retry of a partition throw — persistent corruption in that partition's data region that repositioning cannot fix.

Common situations: Locally destroyed data blocks (bad sectors, corrupted pages); partitions written by a crashing flush that were never fully durable; disk full during compaction truncating writes.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/0b920e0b37c80a65. Report an issue: GitHub.