What causes "Can't revert replacement from " in apache/cassandra?

Check the current step before cancelling; if the replace reached FINISH_REPLACE, treat the node as replaced and use the appropriate removal flow instead

apache/cassandra · error · IllegalStateException

Can't revert replacement from

Error message

Can't revert replacement from

What it means

BootstrapAndReplace.cancel reverts a partially applied replacement by inverting transformations; its switch handles MID_REPLACE/START_REPLACE (falling through to invert startReplace). The default branch throws IllegalStateException when the current step kind is not one of the replace states, so the replacement cannot be reverted from that state.

Source

Thrown at src/java/org/apache/cassandra/tcm/sequences/BootstrapAndReplace.java:333

        InetAddressAndPort replaced = metadata.directory.getNodeAddresses(startReplace.replaced()).broadcastAddress;
        return new ProgressBarrier(latestModification, metadata.directory.location(startReplace.nodeId()), metadata.lockedRanges.locked.get(lockKey), e -> !e.equals(replaced));
    }

    @Override
    public ClusterMetadata.Transformer cancel(ClusterMetadata metadata)
    {
        DataPlacements placements = metadata.placements();
        switch (next)
        {
            // need to undo MID_REPLACE and START_REPLACE, but PREPARE_REPLACE doesn't affect placements
            case FINISH_REPLACE:
                placements = midReplace.inverseDelta().apply(metadata.directory, metadata.nextEpoch(), placements);
            case MID_REPLACE:
            case START_REPLACE:
                placements = startReplace.inverseDelta().apply(metadata.directory, metadata.nextEpoch(), placements);
                break;
            default:
                throw new IllegalStateException("Can't revert replacement from " + next);
        }

        LockedRanges newLockedRanges = metadata.lockedRanges.unlock(lockKey);
        return metadata.transformer()
                       .withNodeState(startReplace.replacement(), NodeState.REGISTERED)
                       .with(placements)
                       .with(newLockedRanges);
    }

    public BootstrapAndReplace finishJoiningRing()
    {
        return new BootstrapAndReplace(latestModification, lockKey, bootstrapTokens,
                                       next, startReplace, midReplace, finishReplace,
                                       true, false);
    }

    /**
     * startDelta.writes.additions contains the ranges we need to stream

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Check the current step before cancelling; if the replace reached FINISH_REPLACE, treat the node as replaced and use the appropriate removal flow instead
Re-read ClusterMetadata for the authoritative node state and re-issue cancellation only for a valid pending state
If the state is genuinely unexpected, treat it as a metadata consistency issue and repair/replay the log
Ensure only one cancellation attempt runs at a time to avoid racing the sequence state

Example fix

// before
bootstrapAndReplace.cancel(metadata);
// after
if (bootstrapAndReplace.nextStep() == Transformation.Kind.FINISH_REPLACE)
    // replacement committed; nothing to revert
else
    bootstrapAndReplace.cancel(metadata);

Defensive patterns

Strategy: try-catch

Validate before calling

Transformation.Kind step = sequence.nextStep();
if (step != Transformation.Kind.START_REPLACE && step != Transformation.Kind.MID_REPLACE)
    throw new IllegalStateException("cannot revert replace from step " + step);

Try / catch

try { seq.cancel(metadata); } catch (IllegalStateException e) { /* if FINISH_REPLACE: node is replaced; use removal flow instead */ }

Prevention

Verify replace state before issuing cancel
Serialize cancellation attempts to avoid races with completion
On committed replacements, switch to decommission/removenode flows

When it happens

Trigger: Calling cancel() on a BootstrapAndReplace whose current step is FINISH_REPLACE (replacement already committed) or any kind outside the replace sequence, e.g. via a retry racing with completion.

Common situations: Operator retries the cancel command after the replacement already finished; a failed replace leaves state at an unexpected kind; metadata replay assigns a non-replace kind to this sequence.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/2897fa98e3738fbd. Report an issue: GitHub.