{"record":{"id":"f4ee42c9db334266","repo":"apache/beam","slug":"some-tokens-are-missing-from-the-splits-this-should-not","errorCode":null,"errorMessage":"Some tokens are missing from the splits. This should not happen.","messagePattern":"Some tokens are missing from the splits\\. This should not happen\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/cassandra/src/main/java/org/apache/beam/sdk/io/cassandra/SplitGenerator.java","lineNumber":141,"sourceCode":"        // See https://issues.apache.org/jira/browse/CASSANDRA-14684\n        endpointTokens.add(\n            token.equals(BigInteger.valueOf(Long.MIN_VALUE)) ? token.add(BigInteger.ONE) : token);\n      }\n\n      // Append the splits between the endpoints\n      for (int j = 0; j < splitCount; j++) {\n        splits.add(RingRange.of(endpointTokens.get(j), endpointTokens.get(j + 1)));\n        LOG.debug(\"Split #{}: [{},{})\", j + 1, endpointTokens.get(j), endpointTokens.get(j + 1));\n      }\n    }\n\n    BigInteger total = BigInteger.ZERO;\n    for (RingRange split : splits) {\n      BigInteger size = split.span(rangeSize);\n      total = total.add(size);\n    }\n    if (!total.equals(rangeSize)) {\n      throw new RuntimeException(\n          \"Some tokens are missing from the splits. \" + \"This should not happen.\");\n    }\n    return coalesceSplits(getTargetSplitSize(totalSplitCount), splits);\n  }\n\n  private boolean isInRange(BigInteger token) {\n    return !(token.compareTo(rangeMin) < 0 || token.compareTo(rangeMax) > 0);\n  }\n\n  private List<List<RingRange>> coalesceSplits(BigInteger targetSplitSize, List<RingRange> splits) {\n    List<List<RingRange>> coalescedSplits = new ArrayList<>();\n    List<RingRange> tokenRangesForCurrentSplit = new ArrayList<>();\n    BigInteger tokenCount = BigInteger.ZERO;\n\n    for (RingRange tokenRange : splits) {\n      if (tokenRange.span(rangeSize).add(tokenCount).compareTo(targetSplitSize) > 0\n          && !tokenRangesForCurrentSplit.isEmpty()) {\n        // enough tokens in that segment","sourceCodeStart":123,"sourceCodeEnd":159,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/cassandra/src/main/java/org/apache/beam/sdk/io/cassandra/SplitGenerator.java#L123-L159","documentation":"After generating splits, generateSplits() asserts that the spans of all produced RingRanges sum exactly to the partitioner's total range size. If not, an internal invariant was violated (tokens lost or double-counted in the split algorithm) and a RuntimeException is thrown.","triggerScenarios":"A bug or inconsistent numeric edge case in split arithmetic — total of split spans != rangeSize — during CassandraIO read split planning.","commonSituations":"Modified/custom SplitGenerator code; integer/precision edge cases around the wrap-around range; unexpected ring data interacting with the algorithm.","solutions":["Report/reproduce with the exact ring token list (nodetool describering) and partitioner.","Verify you are using an unmodified SplitGenerator from your Beam version; upgrade to a newer Beam release.","As a workaround, reduce desired split count or read without explicit split configuration if the issue is split-count dependent."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// pin a Beam version known-good for CassandraIO and avoid custom forked SplitGenerator\nSystem.out.println(org.apache.beam.sdk.Pipeline.class.getPackage().getImplementationVersion());","typeGuard":null,"tryCatchPattern":"try { ... } catch (RuntimeException e) { if (e.getMessage().contains(\"Some tokens are missing\")) { /* report bug with ring dump */ } throw e; }","preventionTips":["Use unmodified upstream SplitGenerator.","Keep Beam up to date.","Capture nodetool describering output for bug reports."],"tags":["java","cassandra","split-generation","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}