apache/cassandra · error · RepairException

Incremental repair session %s has failed

Error message

Incremental repair session %s has failed

What it means

The first of two failure futures set in CoordinatorSession.fail(): when the incremental repair session fails, the coordinator propagates the failure into finalizeProposeFuture and prepareFuture so waiting callers (e.g. repair scheduling) see a RepairException. This index marks the finalizePropose failure.

Source

Thrown at src/java/org/apache/cassandra/repair/consistent/CoordinatorSession.java:324

    }

    public synchronized void fail()
    {
        Set<Map.Entry<InetAddressAndPort, State>> cantFail = participantStates.entrySet()
                                                                              .stream()
                                                                              .filter(entry -> !entry.getValue().canTransitionTo(State.FAILED))
                                                                              .collect(Collectors.toSet());
        if (!cantFail.isEmpty())
        {
            logger.error("Can't transition endpoints {} to FAILED", cantFail, new RuntimeException());
            return;
        }
        logger.info("Incremental repair session {} failed", sessionID);
        sendFailureMessageToParticipants();
        setAll(State.FAILED);

        String exceptionMsg = String.format("Incremental repair session %s has failed", sessionID);
        finalizeProposeFuture.tryFailure(RepairException.warn(exceptionMsg));
        prepareFuture.tryFailure(RepairException.warn(exceptionMsg));
    }

    private static String formatDuration(long then, long now)
    {
        if (then == Long.MIN_VALUE || now == Long.MIN_VALUE)
        {
            // if neither of the times were initially set, don't return a non-sensical answer
            return "n/a";
        }
        return DurationFormatUtils.formatDurationWords(now - then, true, true);
    }

    /**
     * Runs the asynchronous consistent repair session. Actual repair sessions are scheduled via a submitter to make unit testing easier
     */
    public Future<CoordinatedRepairResult> execute(Supplier<Future<CoordinatedRepairResult>> sessionSubmitter)
    {

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Inspect participant node logs for the underlying prepare/finalize failure.
  2. Ensure all replicas of the repaired ranges are up and reachable; re-run repair after restoring them.
  3. Check for concurrent operations blocking prepare (e.g. ongoing repairs on replicas, pending compactions).
  4. Increase repair timeouts if finalize/prepare are timing out under load.
  5. Use nodetool downloadschedules/repair_admin commands (or rerun repair) to clean up failed session state.
Defensive patterns

Strategy: retry

Validate before calling

// Verify all participants up and no prior session active:
// nodetool status
// SELECT * FROM system.repairs WHERE state != 'COMPLETED';

Try / catch

try {
    repair(keyspace, incremental = true);
} catch (RepairException e) {
    if (e.getMessage().matches("Incremental repair session .* has failed")) {
        // check system.repairs for session state; rerun after node recovery
    }
}

Prevention

When it happens

Trigger: fail() is called from handlePrepareResponse (a replica rejected prepare), handleFinalizePromise (a replica rejected finalize), execute (coordinator-side error), or handleFailSessionMessage (a replica reported session failure).

Common situations: Replica down or unreachable mid-session, replica failing prepare due to ongoing operations, validate/finalize timeouts (repair_command_pool_size, cas timeouts), or a participant node crashing during the incremental repair protocol.

Related errors


AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/4d19ab2e77c357e5. Report an issue: GitHub.