apache/cassandra · error · IllegalStateException

Failed setting pending repair to %s on %s (pending repair is

Error message

Failed setting pending repair to %s on %s (pending repair is %s)

What it means

verifyMetadata is the post-condition check after CompactionStrategyManager mutates an sstable's repair metadata (via mutateRepaired, used by repair/anti-compaction to set pendingRepair, repairedAt, and isTransient). It re-reads the live values and throws IllegalStateException if the pendingRepair id written does not match what was requested - i.e. the metadata mutation silently failed or raced with another mutation.

Source

Thrown at src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java:1533

        finally
        {
            try
            {
                // if there was an exception mutating repairedAt, we should still notify for the
                // sstables that we were able to modify successfully before releasing the lock
                cfs.getTracker().notifySSTableRepairedStatusChanged(changed);
            }
            finally
            {
                writeLock.unlock();
            }
        }
    }

    private static void verifyMetadata(SSTableReader sstable, long repairedAt, TimeUUID pendingRepair, boolean isTransient)
    {
        if (!Objects.equals(pendingRepair, sstable.getPendingRepair()))
            throw new IllegalStateException(String.format("Failed setting pending repair to %s on %s (pending repair is %s)", pendingRepair, sstable, sstable.getPendingRepair()));
        if (repairedAt != sstable.getRepairedAt())
            throw new IllegalStateException(String.format("Failed setting repairedAt to %d on %s (repairedAt is %d)", repairedAt, sstable, sstable.getRepairedAt()));
        if (isTransient != sstable.isTransient())
            throw new IllegalStateException(String.format("Failed setting isTransient to %b on %s (isTransient is %b)", isTransient, sstable, sstable.isTransient()));
    }

    public CleanupSummary releaseRepairData(Collection<TimeUUID> sessions)
    {
        List<CleanupTask> cleanupTasks = new ArrayList<>();
        readLock.lock();
        try
        {
            for (PendingRepairManager prm : Iterables.concat(pendingRepairs.getManagers(), transientRepairs.getManagers()))
                cleanupTasks.add(prm.releaseSessionData(sessions));
        }
        finally
        {
            readLock.unlock();

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Re-run repair on the affected ranges - the failed metadata update will be retried safely
Retry repair finalization (repair_admin operations) after transient concurrent activity settles
Avoid running concurrent repairs or manual compactions on the same table during repair finalization
Check logs for which mutation raced; if reproducible without concurrency, file a Cassandra bug

Example fix

// before
sstable.mutateRepaired(ActiveRepairService.UNREPAIRED_SSTABLE, session, false);
// after
try { sstable.mutateRepaired(ActiveRepairService.UNREPAIRED_SSTABLE, session, false); }
catch (IllegalStateException e) { logger.warn("metadata race on {}, will be retried by repair", sstable); }

Defensive patterns

Strategy: try-catch

Validate before calling

// ensure no concurrent repair/compaction before mutating metadata
if (!CompactionManager.instance.getCompactions().isEmpty() || repairInFlight(cfs))
    throw new IllegalStateException("defer metadata mutation until repair/compaction quiesce");

Try / catch

try { sstable.mutateRepairedAt(...); verifyMetadata(sstable, repairedAt, pendingRepair, isTransient); }
catch (IllegalStateException e) {
  logger.warn("repair metadata race on {}", sstable, e);
  // allow repair to retry the finalization
}

Prevention

Serialize repair finalization per table
Do not run user-defined compaction while repair finalizes
Verify metadata only under the sstable's lifecycle lock
Monitor logs for repeated verifyMetadata failures - they indicate real races

When it happens

Trigger: Calling sstable.mutateRepaired/setPendingRepair during incremental repair completion or anticompaction, then verifying; a concurrent metadata mutation (another repair transition, compaction finishing, or sstable being replaced) overwrote pendingRepair between write and read.

Common situations: Concurrent incremental repair sessions touching the same sstable; repair finishing while compaction claims the sstable; races with sstable replacement/mark-compacting during cleanup or repair-finalization; retry storms in repair finalization.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/4df03b91345d9dcc. Report an issue: GitHub.