apache/cassandra · error · IllegalStateException

Discovered existing bootstrap data and

Error message

Discovered existing bootstrap data and %s is not configured; aborting bootstrap. Please clean up local files manually and try again or set cassandra.reset_bootstrap_progress=true to ignore. Found: %s. Fully available: %s. Transiently available: %s

What it means

During bootstrap streaming, RangeStreamer detects existing bootstrap data on local disk while the cassandra.reset_bootstrap_progress flag is not set. To avoid silently mixing incomplete SSTables with new streams, it aborts the bootstrap with this IllegalStateException and asks the operator to either clean the files or set the flag to ignore the previous progress.

Solutions

1. Set -Dcassandra.reset_bootstrap_progress=true (system property) on the node and restart so stale bootstrap progress is ignored
2. Clean the data directories of the affected keyspaces (remove partial SSTables) and restart bootstrap
3. If the node was previously bootstrapped successfully, remove it from the cluster (nodetool decommission) before re-adding
4. Back up data files before deleting them

Example fix

// before
bin/cassandra
// ERROR: Discovered existing bootstrap data and cassandra.reset_bootstrap_progress is not configured...
// after
bin/cassandra -Dcassandra.reset_bootstrap_progress=true
// or: wipe partial data first
rm -rf /var/lib/cassandra/data/*/*(partial bootstrap sstables) && bin/cassandra

// before restart, inspect data dirs for leftover bootstrap sstables
ls /var/lib/cassandra/data/*/*/ | wc -l  # unexpected files from failed bootstrap?

try {
    streamer.fetchAsync(...).get();
} catch (ExecutionException e) {
    if (e.getCause().getMessage().contains("reset_bootstrap_progress")) {
        // either wipe partial data dirs or restart with -Dcassandra.reset_bootstrap_progress=true
    }
}

When it happens

Trigger: Calling fetchAsync during bootstrap when skipped/reused ranges are found available locally and RESET_BOOTSTRAP_PROGRESS=false; typically after a previously interrupted bootstrap attempt.

Common situations: A bootstrap that failed midway leaves partial SSTables in data directories; operator restarts the node and bootstrap reruns; node was never fully decommissioned before re-adding.

Related errors

• A node required to move the data consistently is down
• Can't join the ring because bootstrap hasn't completed.
• Could not finish join for during replacement
• Keyspace is already added to fetch map
• Stream failed: \nSession peer

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/e9df41dd92160d0d. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/dht/RangeStreamer.java:742

                        return true;
                    };

                    remaining = fetchReplicas.stream().filter(not(isAvailable)).collect(Collectors.toList());

                    if (remaining.size() < available.full.size() + available.trans.size())
                    {
                        // If the operator hasn't specified what to do when we discover a previous partially successful bootstrap,
                        // we error out and tell them to manually reconcile it. See CASSANDRA-17679.
                        if (!RESET_BOOTSTRAP_PROGRESS.isPresent())
                        {
                            List<FetchReplica> skipped = fetchReplicas.stream().filter(isAvailable).collect(Collectors.toList());
                            String msg = String.format("Discovered existing bootstrap data and %s " +
                                                       "is not configured; aborting bootstrap. Please clean up local files manually " +
                                                       "and try again or set cassandra.reset_bootstrap_progress=true to ignore. " +
                                                       "Found: %s. Fully available: %s. Transiently available: %s",
                                                       RESET_BOOTSTRAP_PROGRESS.getKey(), skipped, available.full, available.trans);
                            logger.error(msg);
                            throw new IllegalStateException(msg);
                        }

                        if (!RESET_BOOTSTRAP_PROGRESS.getBoolean())
                        {
                            List<FetchReplica> skipped = fetchReplicas.stream().filter(isAvailable).collect(Collectors.toList());
                            logger.info("Some ranges of {} are already available. Skipping streaming those ranges. Skipping {}. Fully available {} Transiently available {}",
                                        fetchReplicas, skipped, available.full, available.trans);
                        }
                    }
                }

                if (logger.isTraceEnabled())
                    logger.trace("{}ing from {} ranges {}", description, source, StringUtils.join(remaining, ", "));

                InetAddressAndPort self = FBUtilities.getBroadcastAddressAndPort();
                RangesAtEndpoint full = remaining.stream()
                                                 .filter(pair -> pair.remote.isFull())
                                                 .map(pair -> pair.local)

View on GitHub (pinned to 88fd0f6a0e)