{"record":{"id":"0b920e0b37c80a65","repo":"apache/cassandra","slug":"retry-failed-too-skipping-to-next-partition-retr-0b920e","errorCode":null,"errorMessage":"Retry failed too. Skipping to next partition (retry's stacktrace follows)","messagePattern":"Retry failed too\\. Skipping to next partition \\(retry's stacktrace follows\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/io/sstable/format/bti/BtiTableScrubber.java","lineNumber":218,"sourceCode":"                    long rowStartFromIndex = dataStartFromIndex + TypeSizes.SHORT_SIZE + currentIndexKey.remaining();\n                    outputHandler.output(\"Retrying from partition index; data is %s bytes starting at %s\",\n                                         dataSizeFromIndex, rowStartFromIndex);\n                    key = sstable.decorateKey(currentIndexKey);\n                    try\n                    {\n                        if (!isIndex)\n                            partitionKeyType.validate(key.getKey());\n                        dataFile.seek(rowStartFromIndex);\n\n                        if (tryAppend(prevKey, key, writer))\n                            prevKey = key;\n                    }\n                    catch (Throwable th2)\n                    {\n                        throwIfFatal(th2);\n                        throwIfCannotContinue(key, th2);\n\n                        outputHandler.warn(th2, \"Retry failed too. Skipping to next partition (retry's stacktrace follows)\");\n                        badPartitions++;\n                        if (!seekToNextPartition())\n                            break;\n                    }\n                }\n                else\n                {\n                    throwIfCannotContinue(key, th);\n\n                    badPartitions++;\n                    if (indexIterator != null)\n                    {\n                        outputHandler.warn(\"Partition starting at position %d is unreadable; skipping to next\", dataStart);\n                        if (!seekToNextPartition())\n                            break;\n                    }\n                    else\n                    {","sourceCodeStart":200,"sourceCodeEnd":236,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/sstable/format/bti/BtiTableScrubber.java#L200-L236","documentation":"After a partition read error, the BTI scrubber repositions the data file using the index and retries the partition once. If that retry also fails, this warning logs the retry's stacktrace, the partition is counted as bad, and the scrubber skips to the next partition (via seekToNextPartition) or aborts if no further partition can be located.","triggerScenarios":"`nodetool scrub` on a BTI sstable where both the original read and the index-guided re-seek+retry of a partition throw — persistent corruption in that partition's data region that repositioning cannot fix.","commonSituations":"Locally destroyed data blocks (bad sectors, corrupted pages); partitions written by a crashing flush that were never fully durable; disk full during compaction truncating writes.","solutions":["Accept the skip if the data is replicated elsewhere; run `nodetool repair` to re-replicate the lost partitions.","Inspect the logged stacktrace for the root cause (IO error vs malformed data).","If badPartitions is nonzero and data is unrecoverable locally, restore the affected sstables from a snapshot.","Fix the underlying storage issue (disk errors, permissions) before scrubbing again."],"exampleFix":"// Recover data from replicas:\n// before: scrub skips partition, local data lost\n// after:\n$ nodetool repair <keyspace> <table>\n// verify with:\n$ nodetool verify <keyspace> <table>","handlingStrategy":"retry","validationCode":"// Verify replicas hold the data before accepting skipped partitions\n$ nodetool repair --dry-run keyspace table","typeGuard":null,"tryCatchPattern":"// Count bad partitions post-scrub and trigger repair if any\nint badPartitions = scrubResult.getBadPartitions();\nif (badPartitions > 0) { /* run nodetool repair */ }","preventionTips":["Maintain replication factor >= 3 so skipped partitions are recoverable from replicas.","Run nodetool repair after any scrub that reports bad partitions.","Replace failing hardware promptly; repeated retry failures indicate real disk damage.","Avoid filling disks > 80% capacity, which leads to truncated writes."],"tags":["cassandra","scrub","sstable","data-corruption"],"backgroundTag":"checksum-mismatch","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}