apache/cassandra · warning

Growth of %.2f%% in token ownership standard deviation…

Error message

Growth of %.2f%% in token ownership standard deviation after allocation above warning threshold of %d%%

What it means

A logger.warn, not an exception: TokenAllocation.allocate() compares ownership distribution standard deviation before and after allocating new tokens and warns when the growth exceeds WARN_STDEV_GROWTH (10%). It means the newly allocated tokens increased load imbalance among replicated nodes in the datacenter beyond the tolerated amount.

Solutions

1. Check cluster balance with `nodetool tablehistograms` / ownership before adding the node; rebalance first if already skewed.
2. Use allocate_tokens_for_local_replication_factor in cassandra.yaml so allocation accounts for the full RF.
3. Ensure rackId/DC configuration of the bootstrapping node is correct so allocation targets the right rack.
4. If the warning is expected (deliberate imbalance), it can be acknowledged; it does not abort bootstrap.

Example fix

// before: bootstrap without RF-aware allocation
initial_token:
auto_bootstrap: true
// after: cassandra.yaml
allocate_tokens_for_local_replication_factor: 5  // allocation minimizes stdev growth for RF=5

// verify pre-allocation balance with nodetool or OwnershipStats
OwnershipStats os = TokenAllocation.replicatedOwnership(metadata, snitch, ks, rf);
if (os.getStandardDeviation() > 0.10) log.warn("Cluster already imbalanced; consider rebalancing first");

When it happens

Trigger: Calling TokenAllocation.tokens()/allocate() (used by NetworkTopologyStrategy token allocation when a node bootstraps with allocate_tokens_for_local_replication_factor or similar) where post-allocation ownership stdev grows >10% versus pre-allocation.

Common situations: Bootstrapping a node into a datacenter with pre-existing ownership skew; allocating tokens for a node into a rack that is already over-loaded; misuse of replication factor settings causing wrong replicated-load baseline.

Related errors

• Growth of %.2f%% in token ownership standard deviation…
• manually specified tokens override automatic allocation
• Picking random token for a single vnode. You should…
• Problem opening token allocation keyspace " +…
• A node required to move the data consistently is down

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/1926967013e952c6. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/dht/tokenallocator/TokenAllocation.java:134

    Collection<Token> allocate(InetAddressAndPort endpoint)
    {
        StrategyAdapter strategy = getOrCreateStrategy(endpoint);
        Collection<Token> tokens = strategy.createAllocator().addUnit(endpoint, numTokens);
        tokens = strategy.adjustForCrossDatacenterClashes(tokens);

        SummaryStatistics os = strategy.replicatedOwnershipStats();
        NodeId nodeId = metadata.directory.peerId(endpoint);
        updateTokensForNode(nodeId, tokens);

        SummaryStatistics ns = strategy.replicatedOwnershipStats();
        logger.info("Selected tokens {}", tokens);
        logger.debug("Replicated node load in datacenter before allocation {}", statToString(os));
        logger.debug("Replicated node load in datacenter after allocation {}", statToString(ns));

        double stdDevGrowth = ns.getStandardDeviation() - os.getStandardDeviation();
        if (stdDevGrowth > TokenAllocation.WARN_STDEV_GROWTH)
        {
            logger.warn(String.format("Growth of %.2f%% in token ownership standard deviation after allocation above warning threshold of %d%%",
                                      stdDevGrowth * 100, (int)(TokenAllocation.WARN_STDEV_GROWTH * 100)));
        }

        return tokens;
    }

    static String statToString(SummaryStatistics stat)
    {
        return String.format("max %.2f min %.2f stddev %.4f", stat.getMax() / stat.getMean(), stat.getMin() / stat.getMean(), stat.getStandardDeviation());
    }

    SummaryStatistics getAllocationRingOwnership(String datacenter, String rack)
    {
        return getOrCreateStrategy(datacenter, rack).replicatedOwnershipStats();
    }

    SummaryStatistics getAllocationRingOwnership(InetAddressAndPort endpoint)
    {

View on GitHub (pinned to 88fd0f6a0e)