apache/cassandra · error · IllegalStateException

Discovered existing bootstrap data and %s is not configured;

Error message

Discovered existing bootstrap data and %s is not configured; aborting bootstrap. Please clean up local files manually and try again or set cassandra.reset_bootstrap_progress=true to ignore. Found: %s. Fully available: %s. Transiently available: %s

What it means

During bootstrap streaming, RangeStreamer detects existing bootstrap data on local disk while the cassandra.reset_bootstrap_progress flag is not set. To avoid silently mixing incomplete SSTables with new streams, it aborts the bootstrap with this IllegalStateException and asks the operator to either clean the files or set the flag to ignore the previous progress.

Source

Thrown at src/java/org/apache/cassandra/dht/RangeStreamer.java:742

                        return true;
                    };

                    remaining = fetchReplicas.stream().filter(not(isAvailable)).collect(Collectors.toList());

                    if (remaining.size() < available.full.size() + available.trans.size())
                    {
                        // If the operator hasn't specified what to do when we discover a previous partially successful bootstrap,
                        // we error out and tell them to manually reconcile it. See CASSANDRA-17679.
                        if (!RESET_BOOTSTRAP_PROGRESS.isPresent())
                        {
                            List<FetchReplica> skipped = fetchReplicas.stream().filter(isAvailable).collect(Collectors.toList());
                            String msg = String.format("Discovered existing bootstrap data and %s " +
                                                       "is not configured; aborting bootstrap. Please clean up local files manually " +
                                                       "and try again or set cassandra.reset_bootstrap_progress=true to ignore. " +
                                                       "Found: %s. Fully available: %s. Transiently available: %s",
                                                       RESET_BOOTSTRAP_PROGRESS.getKey(), skipped, available.full, available.trans);
                            logger.error(msg);
                            throw new IllegalStateException(msg);
                        }

                        if (!RESET_BOOTSTRAP_PROGRESS.getBoolean())
                        {
                            List<FetchReplica> skipped = fetchReplicas.stream().filter(isAvailable).collect(Collectors.toList());
                            logger.info("Some ranges of {} are already available. Skipping streaming those ranges. Skipping {}. Fully available {} Transiently available {}",
                                        fetchReplicas, skipped, available.full, available.trans);
                        }
                    }
                }

                if (logger.isTraceEnabled())
                    logger.trace("{}ing from {} ranges {}", description, source, StringUtils.join(remaining, ", "));

                InetAddressAndPort self = FBUtilities.getBroadcastAddressAndPort();
                RangesAtEndpoint full = remaining.stream()
                                                 .filter(pair -> pair.remote.isFull())
                                                 .map(pair -> pair.local)

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Set -Dcassandra.reset_bootstrap_progress=true (system property) on the node and restart so stale bootstrap progress is ignored
2. Clean the data directories of the affected keyspaces (remove partial SSTables) and restart bootstrap
3. If the node was previously bootstrapped successfully, remove it from the cluster (nodetool decommission) before re-adding
4. Back up data files before deleting them

Example fix

// before
bin/cassandra
// ERROR: Discovered existing bootstrap data and cassandra.reset_bootstrap_progress is not configured...
// after
bin/cassandra -Dcassandra.reset_bootstrap_progress=true
// or: wipe partial data first
rm -rf /var/lib/cassandra/data/*/*(partial bootstrap sstables) && bin/cassandra

// before restart, inspect data dirs for leftover bootstrap sstables
ls /var/lib/cassandra/data/*/*/ | wc -l  # unexpected files from failed bootstrap?

try {
    streamer.fetchAsync(...).get();
} catch (ExecutionException e) {
    if (e.getCause().getMessage().contains("reset_bootstrap_progress")) {
        // either wipe partial data dirs or restart with -Dcassandra.reset_bootstrap_progress=true
    }
}

When it happens

Trigger: Calling fetchAsync during bootstrap when skipped/reused ranges are found available locally and RESET_BOOTSTRAP_PROGRESS=false; typically after a previously interrupted bootstrap attempt.

Common situations: A bootstrap that failed midway leaves partial SSTables in data directories; operator restarts the node and bootstrap reruns; node was never fully decommissioned before re-adding.

Related errors

• Unable to find sufficient sources for streaming range " + tr
• Unable to find sufficient sources for streaming range " + ra
• Keyspace is already added to fetch map
• Unable to find sufficient sources for streaming range in ke
• Could not finish join for during replacement

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/e9df41dd92160d0d. Report an issue: GitHub.