apache/cassandra · critical · java.lang.RuntimeException

HostId read from local system table does not match the one r

Error message

HostId read from local system table does not match the one recorded for this endpoint during initial cluster metadata conversion. Endpoint: %s, NodeId: %s, Recorded: %s, Local: %s

What it means

Thrown during upgrade-time registration when the host id stored in the node's local system.local does not match the host id recorded for this node's address in the initial cluster metadata conversion. The mismatch means the local node state and CMS directory disagree about node identity, which would corrupt membership tracking.

Source

Thrown at src/java/org/apache/cassandra/tcm/transformations/Register.java:198

                throw new IllegalStateException("Node has host id "+localHostId+" in system.local, but is not present in cluster metadata - not allowing this node to register. " +
                                                "If a bootstrap of this node failed and was aborted with `nodetool abortbootstrap` it should also have its data removed before trying to rebootstrap.");
            NodeVersion dirVersion = directory.version(nodeId);

            // If this is a node in the process of upgrading, update the host id in the system.local table
            // TODO: when constructing the initial cluster metadata for upgrade, we include a mapping from
            //      NodeId to the old HostId. We will need to use this lookup to map between the two for
            //      hint delivery immediately following an upgrade.
            if (dirVersion == null || !dirVersion.isUpgraded())
            {
                if (directory.hostId(nodeId).equals(localHostId))
                {
                    SystemKeyspace.setLocalHostId(nodeId.toUUID());
                    logger.info("Updated local HostId from pre-upgrade version {} to the one which was pre-registered " +
                                "during initial cluster metadata conversion {}", localHostId, nodeId.toUUID());
                }
                else
                {
                    throw new RuntimeException("HostId read from local system table does not match the one recorded " +
                                               "for this endpoint during initial cluster metadata conversion. " +
                                               String.format("Endpoint: %s, NodeId: %s, Recorded: %s, Local: %s",
                                                             FBUtilities.getBroadcastAddressAndPort(),
                                                             nodeId,
                                                             directory.hostId(nodeId),
                                                             localHostId));
                }
            }
            else
            {
                logger.info("Local id was already registered, retaining: {}", localHostId);
            }
            return nodeId;
        }
    }

    @Override
    public String toString()

View on GitHub (pinned to 88fd0f6a0e)

Solutions

Correct system.local host id to match the id recorded in cluster metadata (restore proper node data)
Verify you did not copy another node's data directory onto this host; restore this node's own data
If metadata is wrong, fix the CMS directory entry (cms tool) or remove and re-register the node

Example fix

// before: restored wrong snapshot, system.local hostId=uuid-A but CMS says uuid-B
// after: restore the node's own data so system.local hostId matches directory.hostId(nodeId),
// or run: cqlsh> UPDATE system.local SET host_id = <uuid-from-cms> WHERE key='local';

Defensive patterns

Strategy: validation

Validate before calling

// before upgrade registration, compare ids
UUID localId = SystemKeyspace.getLocalHostId();
UUID recordedId = directory.hostId(directory.peerId(broadcastAddress));
if (localId != null && recordedId != null && !localId.equals(recordedId)) { /* fix before proceeding */ }

Try / catch

try { register(); } catch (RuntimeException e) { if (e.getMessage().startsWith("HostId read from local system table does not match")) { /* restore correct node data or fix CMS entry */ } else throw e; }

Prevention

Never copy one node's data directory to another host
During upgrades, back up system.local per node and restore only the node's own backup
Cross-check host ids against CMS after any data restore

When it happens

Trigger: register() during upgrade finds directory.peerId(broadcastAddress) non-null, but directory.hostId(nodeId) differs from the localHostId read from system.local (and it is not the pre-upgrade path that would update system.local).

Common situations: Upgrading to TCM-based metadata after system.local was regenerated from the wrong backup; cloning a node's data directory onto another node; mixed/incorrect restore procedures.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/e5892506b4afb113. Report an issue: GitHub.