apache/cassandra · warning

Locations differ: !=

Error message

Locations differ: {} != {}

What it means

WARN log from Directory.dumpDiff(Directory) indicating the locations maps (NodeId to DataCenter/location placement) differ between two Directory snapshots. Placement data divergence means the two cluster views would compute different token ownership and replication placement. Diagnostic only.

Solutions

  1. Compare which node IDs have differing locations and re-apply the intended topology change as a committed TCM transformation
  2. Replay the cluster metadata log on the diverging node until locations converge
  3. Ensure snitch-based dc/rack assignments are consistent across the cluster before committing topology changes
  4. If a snapshot is simply stale, re-fetch the current Directory from the CMS before diffing
Defensive patterns

Strategy: validation

Validate before calling

// Validate intended topology is committed cluster-wide before diffing
ClusterMetadata.sync(nodes, targetEpoch);
for (Node node : nodes)
    if (!node.directory.locations().equals(expectedLocations))
        System.out.println(node + " has stale locations");

Prevention

When it happens

Trigger: dumpDiff(other) called when the locations maps are unequal, typically after a node moved datacenters in one view (e.g. via a Move/Locate transformation) or a node is present in one map but missing in the other.

Common situations: Operator corrected a node's dc/rack on one node only; directory snapshots compared across a topology migration; stale node replayed older metadata where placements had not yet been updated.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/bca89d2ebcf0b8c0. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/tcm/membership/Directory.java:912

    public void dumpDiff(Directory other)
    {
        if (nextId != other.nextId)
        {
            logger.warn("nextId differ: {} != {}", nextId, other.nextId);
        }
        if (!Objects.equals(lastModified, other.lastModified))
        {
            logger.warn("Last modified differ: {} != {}", lastModified, other.lastModified);
        }
        if (!Objects.equals(peers, other.peers))
        {
            logger.warn("Peers differ: {} != {}", peers, other.peers);
            dumpDiff(logger, peers, other.peers);
        }
        if (!Objects.equals(locations, other.locations))
        {
            logger.warn("Locations differ: {} != {}", locations, other.locations);
        }
        if (!Objects.equals(states, other.states))
        {
            logger.warn("States differ: {} != {}", states, other.states);
            dumpDiff(logger, states, other.states);
        }
        if (!Objects.equals(endpointsByDC, other.endpointsByDC))
        {
            logger.warn("Endpoints by dc differ: {} != {}", endpointsByDC, other.endpointsByDC);
            dumpDiff(logger, endpointsByDC.asMap(), other.endpointsByDC.asMap());
        }
        if (!Objects.equals(racksByDC, other.racksByDC))
        {
            logger.warn("Racks by dc differ: {} != {}", racksByDC, other.racksByDC);
        }
        if (!Objects.equals(versions, other.versions))
        {
            logger.warn("Versions differ: {} != {}", versions, other.versions);

View on GitHub (pinned to 88fd0f6a0e)