apache/cassandra · critical · IllegalStateException

When reinitializing with cluster metadata, we must be in the

Error message

When reinitializing with cluster metadata, we must be in the CMS

What it means

When reinitializing with a serialized ClusterMetadata snapshot, Startup.reinitializeWithClusterMetadata requires the local node to be a CMS member of that metadata (metadata.isCMSMember()). If the node is not part of the CMS in the snapshot, it throws IllegalStateException 'When reinitializing with cluster metadata, we must be in the CMS'. This reinitialization path is only valid for nodes that were CMS members; a non-CMS node cannot reinitialize from this snapshot.

Source

Thrown at src/java/org/apache/cassandra/tcm/Startup.java:601

    public static void reinitializeWithClusterMetadata(String fileName, Function<Processor, Processor> wrapProcessor, Runnable initMessaging) throws IOException, StartupException
    {
        ClusterMetadata prev = ClusterMetadata.currentNullable();
        // First set a minimal ClusterMetadata as some deserialization depends
        // on ClusterMetadata.current() to access the partitioner
        StubClusterMetadataService initial = StubClusterMetadataService.forClientTools();
        ClusterMetadataService.unsetInstance();
        StubClusterMetadataService.setInstance(initial);

        ClusterMetadata metadata = ClusterMetadataService.deserializeClusterMetadata(fileName);
        // if the partitioners are mismatching, we probably won't even get this far
        if (metadata.partitioner != DatabaseDescriptor.getPartitioner())
            throw new IllegalStateException(String.format("When reinitializing with cluster metadata, the same " +
                                                          "partitioner must be used. Configured: %s, Serialized: %s",
                                                          DatabaseDescriptor.getPartitioner().getClass().getCanonicalName(),
                                                          metadata.partitioner.getClass().getCanonicalName()));

        if (!metadata.isCMSMember())
            throw new IllegalStateException("When reinitializing with cluster metadata, we must be in the CMS");

        metadata = metadata.forceEpoch(metadata.epoch.nextEpoch());
        ClusterMetadataService.unsetInstance();
        LocalLog.LogSpec logSpec = LocalLog.logSpec()
                                           .afterReplay(Startup::scrubDataDirectories,
                                                        (_metadata) -> StorageService.instance.registerMBeans())
                                           .withPreviousState(prev)
                                           .withInitialState(metadata)
                                           .withStorage(LogStorage.SystemKeyspace)
                                           .withDefaultListeners()
                                           .isReset(true);

        ClusterMetadataService.setInstance(new ClusterMetadataService(new UniformRangePlacement(),
                                                                      wrapProcessor,
                                                                      ClusterMetadataService::state,
                                                                      logSpec));

        ClusterMetadataService.instance().log().ready();

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Confirm this node is (and was at snapshot time) a CMS member; the reinitialize-with-metadata path only applies to CMS nodes.
If this is a non-CMS node, use the normal discovery/startup path (initializeForDiscovery / initializeAsNonCmsNode) instead.
Take a fresh metadata snapshot from the correct (CMS-member) node if the current one is from the wrong source.
Rebuild this node as a regular member and, if needed, promote it to CMS afterwards with the CMS tooling.

Defensive patterns

Strategy: validation

Validate before calling

// before reinitializing, check this node is a CMS member of the snapshot
if (!metadata.isCMSMember())
    throw new ConfigurationException("Snapshot is not for a CMS-member node; use standard startup");

Try / catch

try { Startup.reinitializeWithClusterMetadata(fileName); }
catch (IllegalStateException e) {
    if ("When reinitializing with cluster metadata, we must be in the CMS".equals(e.getMessage())) {
        // fall back to normal discovery/startup path for non-CMS nodes
    } else throw e;
}

Prevention

Only use the reinitialize path on nodes confirmed as CMS members
Tag snapshots with the source node's id/role
Document recovery procedures per node role (CMS vs non-CMS)
Verify snapshot provenance before restore

When it happens

Trigger: reinitializeWithClusterMetadata loads the serialized ClusterMetadata and metadata.isCMSMember() returns false — the node's id is not among the CMS members recorded in the snapshot; the snapshot came from a different node or the node was removed from CMS before the snapshot was taken.

Common situations: Restoring a metadata dump onto a node that was never (or no longer) a CMS member; snapshot taken before this node joined the CMS; operator using the reinitialize path intended only for CMS nodes during recovery.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/bb8d5baa87faaaaa. Report an issue: GitHub.