apache/beam · error · RuntimeException

Tokens ( , ): two nodes have the same token

Error message

Tokens (%s,%s): two nodes have the same token

What it means

generateSplits() throws when two adjacent ring entries own the same token, i.e. duplicate tokens in the ring, unless the ring has exactly one token range. This indicates corrupt/incoherent ring topology for split computation.

Solutions

1. Run nodetool ring/ describering and find nodes with duplicate tokens.
2. Rebalance: move or decommission the node holding a duplicated token.
3. Prefer virtual nodes (num_tokens > 1) to avoid manual initial_token collisions.
4. Refresh ring metadata / restart drivers to clear stale ring views.

Set<BigInteger> tokens = new HashSet<>(ringTokens);
if (tokens.size() != ringTokens.size()) throw new IllegalStateException("Duplicate ring tokens detected");

try { ... } catch (RuntimeException e) { if (e.getMessage().contains("same token")) { /* rebalance cluster */ } throw e; }

When it happens

Trigger: CassandraIO read where the ring token list contains two nodes with identical token values and tokenRangeCount != 1.

Common situations: Duplicate token assignment in an old-style (vnode-less) cluster where initial_token was manually set identically on two nodes; bootstrap/decommission glitches leaving stale ring entries.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Tokens ( , ) not in range of
• Some tokens are missing from the splits. This should not…
• Unsupported partitioner. Only Random and Murmur3 are…
• 2xx codes should not be exceptions. Got status code
• A 'datagen' table requires either 'rows-per-second' (for…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/a14dbd41628413b2. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/cassandra/src/main/java/org/apache/beam/sdk/io/cassandra/SplitGenerator.java:92

   * @param totalSplitCount requested total amount of splits. This function may generate more
   *     splits.
   * @param ringTokens list of all start tokens in big0 cluster. They have to be in ring order.
   * @return big0 list containing at least {@code totalSplitCount} splits.
   */
  List<List<RingRange>> generateSplits(long totalSplitCount, List<BigInteger> ringTokens) {
    int tokenRangeCount = ringTokens.size();

    List<RingRange> splits = new ArrayList<>();
    for (int i = 0; i < tokenRangeCount; i++) {
      BigInteger start = ringTokens.get(i);
      BigInteger stop = ringTokens.get((i + 1) % tokenRangeCount);

      if (!isInRange(start) || !isInRange(stop)) {
        throw new RuntimeException(
            String.format("Tokens (%s,%s) not in range of %s", start, stop, partitioner));
      }
      if (start.equals(stop) && tokenRangeCount != 1) {
        throw new RuntimeException(
            String.format("Tokens (%s,%s): two nodes have the same token", start, stop));
      }

      BigInteger rs = stop.subtract(start);
      if (rs.compareTo(BigInteger.ZERO) <= 0) {
        // wrap around case
        rs = rs.add(rangeSize);
      }

      // the below, in essence, does this:
      // splitCount = ceiling((rangeSize / RANGE_SIZE) * totalSplitCount)
      BigInteger[] splitCountAndRemainder =
          rs.multiply(BigInteger.valueOf(totalSplitCount)).divideAndRemainder(rangeSize);

      int splitCount =
          splitCountAndRemainder[0].intValue()
              + (splitCountAndRemainder[1].equals(BigInteger.ZERO) ? 0 : 1);

View on GitHub (pinned to 12126d8942)