{"record":{"id":"cff445023e5209ab","repo":"apache/cassandra","slug":"skipped-batch-replay-of-due-to","errorCode":null,"errorMessage":"Skipped batch replay of {} due to {}","messagePattern":"Skipped batch replay of (.+?) due to (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/batchlog/BatchlogManager.java","lineNumber":315,"sourceCode":"        ArrayList<ReplayingBatch> unfinishedBatches = new ArrayList<>(pageSize);\n\n        Set<UUID> hintedNodes = new HashSet<>();\n        Set<TimeUUID> replayedBatches = new HashSet<>();\n        Exception caughtException = null;\n        int skipped = 0;\n\n        // Sending out batches for replay without waiting for them, so that one stuck batch doesn't affect others\n        for (UntypedResultSet.Row row : batches)\n        {\n            TimeUUID id = row.getTimeUUID(\"id\");\n            int version = row.getInt(\"version\");\n            try\n            {\n                dispatchBatch(rateLimiter, row, id, version, hintedNodes, unfinishedBatches);\n            }\n            catch (IOException e)\n            {\n                logger.warn(\"Skipped batch replay of {} due to {}\", id, e.getMessage());\n                caughtException = e;\n                remove(id);\n                ++skipped;\n            }\n\n            if (++positionInPage == pageSize)\n            {\n                // We have reached the end of a batch. To avoid keeping more than a page of mutations in memory,\n                // finish processing the page before requesting the next row.\n                finishAndClearBatches(unfinishedBatches, hintedNodes, replayedBatches);\n                positionInPage = 0;\n            }\n        }\n\n        // finalize the incomplete last page of batches\n        if (positionInPage > 0)\n            finishAndClearBatches(unfinishedBatches, hintedNodes, replayedBatches);\n        else","sourceCodeStart":297,"sourceCodeEnd":333,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/batchlog/BatchlogManager.java#L297-L333","documentation":"During batchlog replay, BatchlogManager attempts to dispatch each stored batch (reading its mutations and sending hinted deliveries). If dispatch throws an IOException, that single batch is skipped: it is removed from the batchlog, counted, and this warning records which batch id was skipped and why. The first exception is retained and re-logged at the end of the page pass.","triggerScenarios":"dispatchBatch raises IOException for a specific batch — typically a corrupted or oversized batch payload in the batchlog table, deserialization failure of stored mutations, or I/O errors reading the row.","commonSituations":"Corrupt SSTable data in system.batches after unclean shutdown or disk issues; batch written by a node running an incompatible serialization version being replayed after an upgrade/downgrade; extremely large batches exceeding limits.","solutions":["Run `nodetool scrub system.batches` (or a full repair) to remove/repair corrupt batchlog rows.","Check disk health and logs around the skip for the underlying IOException message included in the summary warning.","If it followed a version upgrade/downgrade, let the affected batches drop (they were already delivered or are lost) and verify client writers resend as needed.","Confirm cluster versions are consistent (nodetool version on all nodes) to avoid cross-version deserialization problems."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Operators: watch for the trailing aggregated warning and per-batch ids\n// if skips recur across replays, scrub the table:\n//   nodetool scrub system.batches\n// application side: use idempotent writes / LIGHTWEIGHT_TRANSACTIONS or re-send missed mutations","preventionTips":["Keep cluster node versions homogeneous to avoid cross-version deserialization of batchlog entries.","Monitor disk health; most skips trace back to corrupt SSTables or I/O errors.","After unclean shutdowns, verify system.batches integrity and scrub if warnings appear.","Design writers to tolerate lost batchlog entries (application-level retries)."],"tags":["batchlog","io","data-corruption","replay"],"backgroundTag":"batch-replay-skipped","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}