apache/cassandra · critical · RuntimeException

Cannot replace with a node that is already bootstrapped

Error message

Cannot replace with a node that is already bootstrapped

What it means

When starting in replace mode (cassandra.replace_address set), the replacing node must be new: if system tables show bootstrap already completed on this node, replacing would destroy a node that is a legitimate ring member. Cassandra refuses with a RuntimeException.

Solutions

1. Remove -Dcassandra.replace_address from startup options and restart normally
2. If a fresh replacement is truly intended, wipe the data directories so bootstrap state is cleared
3. Verify replace flags are cleared from config before any planned restart

Example fix

// before
JVM_OPTS="$JVM_OPTS -Dcassandra.replace_address=10.0.0.5"  # left over
// after
# remove the line and restart; node rejoins as itself

if (Boolean.getBoolean("cassandra.replace_address") && SystemKeyspace.bootstrapComplete()) throw new IllegalStateException("remove replace flag");

try { startNode(); } catch (RuntimeException e) { if (e.getMessage().contains("already bootstrapped")) { /* remove replace_address and restart */ } }

When it happens

Trigger: Calling initServer with cassandra.replace_address configured while SystemKeyspace.bootstrapComplete() returns true — the local node has previously finished bootstrapping.

Common situations: Leaving cassandra.replace_address in jvm-server.options after a successful replace and restarting the node; accidentally reusing a bootstrap node's data dir as a replacement.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Cannot replace node which is not currently joined
• manually specified tokens override automatic allocation
• num_tokens must be >= 1
• Picking random token for a single vnode. You should…
• Problem opening token allocation keyspace " +…

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/57507fe905d020e6. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/service/StorageService.java:813

        if (SystemKeyspace.wasDecommissioned())
        {
            if (CassandraRelevantProperties.OVERRIDE_DECOMMISSION.getBoolean())
            {
                logger.warn("This node was decommissioned, but overriding by operator request.");
            }
            else
            {
                throw new ConfigurationException("This node was decommissioned and will not rejoin the ring unless cassandra.override_decommission=true has been set, or all existing data is removed and the node is bootstrapped again");
            }
        }

        if (DatabaseDescriptor.getReplaceTokens().size() > 0 || DatabaseDescriptor.getReplaceNode() != null)
            throw new RuntimeException("Replace method removed; use cassandra.replace_address instead");

        if (isReplacing())
        {
            if (SystemKeyspace.bootstrapComplete())
                throw new RuntimeException("Cannot replace with a node that is already bootstrapped");

            InetAddressAndPort replaceAddress = DatabaseDescriptor.getReplaceAddress();
            Directory directory = ClusterMetadata.current().directory;
            if (directory.peerId(replaceAddress) == null || directory.peerState(replaceAddress) != JOINED)
                throw new RuntimeException(String.format("Cannot replace node %s which is not currently joined", replaceAddress));

            BootstrapAndReplace.checkUnsafeReplace(shouldBootstrap());
        }

        if (isReplacingSameAddress())
        {
            BootstrapAndReplace.gossipStateToHibernate(ClusterMetadata.current(), ClusterMetadata.currentNullable().myNodeId());
            Gossiper.instance.start(SystemKeyspace.incrementAndGetGeneration(), false);
        }
        else
        {
            Gossiper.instance.start(SystemKeyspace.incrementAndGetGeneration(),
                                    ClusterMetadataService.state() != ClusterMetadataService.State.GOSSIP); // only populate local state if not running in gossip mode

View on GitHub (pinned to 88fd0f6a0e)