What causes "Skipped batch replay of {} due to {}" in apache/cassandra?

Run `nodetool scrub system.batches` (or a full repair) to remove/repair corrupt batchlog rows.

Skipped batch replay of

Error message

Skipped batch replay of {} due to {}

What it means

During batchlog replay, BatchlogManager attempts to dispatch each stored batch (reading its mutations and sending hinted deliveries). If dispatch throws an IOException, that single batch is skipped: it is removed from the batchlog, counted, and this warning records which batch id was skipped and why. The first exception is retained and re-logged at the end of the page pass.

Solutions

Run `nodetool scrub system.batches` (or a full repair) to remove/repair corrupt batchlog rows.
Check disk health and logs around the skip for the underlying IOException message included in the summary warning.
If it followed a version upgrade/downgrade, let the affected batches drop (they were already delivered or are lost) and verify client writers resend as needed.
Confirm cluster versions are consistent (nodetool version on all nodes) to avoid cross-version deserialization problems.

Defensive patterns

Strategy: retry

Try / catch

// Operators: watch for the trailing aggregated warning and per-batch ids
// if skips recur across replays, scrub the table:
//   nodetool scrub system.batches
// application side: use idempotent writes / LIGHTWEIGHT_TRANSACTIONS or re-send missed mutations

Prevention

Keep cluster node versions homogeneous to avoid cross-version deserialization of batchlog entries.
Monitor disk health; most skips trace back to corrupt SSTables or I/O errors.
After unclean shutdowns, verify system.batches integrity and scrub if warnings appear.
Design writers to tolerate lost batchlog entries (application-level retries).

When it happens

Trigger: dispatchBatch raises IOException for a specific batch — typically a corrupted or oversized batch payload in the batchlog table, deserialization failure of stored mutations, or I/O errors reading the row.

Common situations: Corrupt SSTable data in system.batches after unclean shutdown or disk issues; batch written by a node running an incompatible serialization version being replayed after an upgrade/downgrade; extremely large batches exceeding limits.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/cff445023e5209ab. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/batchlog/BatchlogManager.java:315

        ArrayList<ReplayingBatch> unfinishedBatches = new ArrayList<>(pageSize);

        Set<UUID> hintedNodes = new HashSet<>();
        Set<TimeUUID> replayedBatches = new HashSet<>();
        Exception caughtException = null;
        int skipped = 0;

        // Sending out batches for replay without waiting for them, so that one stuck batch doesn't affect others
        for (UntypedResultSet.Row row : batches)
        {
            TimeUUID id = row.getTimeUUID("id");
            int version = row.getInt("version");
            try
            {
                dispatchBatch(rateLimiter, row, id, version, hintedNodes, unfinishedBatches);
            }
            catch (IOException e)
            {
                logger.warn("Skipped batch replay of {} due to {}", id, e.getMessage());
                caughtException = e;
                remove(id);
                ++skipped;
            }

            if (++positionInPage == pageSize)
            {
                // We have reached the end of a batch. To avoid keeping more than a page of mutations in memory,
                // finish processing the page before requesting the next row.
                finishAndClearBatches(unfinishedBatches, hintedNodes, replayedBatches);
                positionInPage = 0;
            }
        }

        // finalize the incomplete last page of batches
        if (positionInPage > 0)
            finishAndClearBatches(unfinishedBatches, hintedNodes, replayedBatches);
        else

View on GitHub (pinned to 88fd0f6a0e)