stanfordnlp/CoreNLP · error · RuntimeException

Tried to compare two Distribution<K> objects but d1.counter

Error message

Tried to compare two Distribution<K> objects but d1.counter intersect d2.counter > numberOfKeys

What it means

After taking the union of the two distributions' key sets, getSetOfAllKeys() checks an invariant: the union size must not exceed d1's numberOfKeys, because all keys of both distributions must belong to the shared universe of that size. A larger union proves the distributions were built with inconsistent numberOfKeys or contain keys outside the declared universe, so a RuntimeException is thrown (the message says 'intersect' but the check is on the union).

Solutions

1. Increase numberOfKeys on both distributions to at least the size of the true combined key universe
2. Rebuild both distributions consistently from the full key space
3. Compare via your own union code if the invariant of shared numberOfKeys does not hold for your data

Example fix

// before
Distribution<String> d1 = new Distribution<>(c1, 10); // too small
Distribution<String> d2 = new Distribution<>(c2, 10);
double j = Distributions.jaccardCoefficient(d1, d2); // union of keys > 10
// after
Set<String> universe = union(c1.keySet(), c2.keySet());
Distribution<String> d1 = new Distribution<>(c1, universe.size());
Distribution<String> d2 = new Distribution<>(c2, universe.size());

Set<K> union = new HashSet<>(d1.getCounter().keySet());
union.addAll(d2.getCounter().keySet());
if (union.size() > d1.getNumberOfKeys()) {
  throw new IllegalStateException("union of keys exceeds numberOfKeys=" + d1.getNumberOfKeys());
}

try {
  double overlap = Distributions.overlap(d1, d2);
} catch (RuntimeException e) {
  if (e.getMessage().contains("intersect d2.counter")) { /* fix numberOfKeys declarations */ }
}

When it happens

Trigger: Comparing two distributions where the combined distinct keys exceed d1.getNumberOfKeys() — e.g. numberOfKeys was under-declared, or the distributions were built over genuinely different key spaces with the same declared count.

Common situations: Under-reporting numberOfKeys when constructing distributions manually (Distributionconstructor with explicit numberOfKeys) then comparing via Distributions.overlap / intersection / jaccardCoefficient.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Tried to compare two Distribution<K> objects but d1.numberOf
• This point should never be reached
• oldTag starts with B, entity at position should not be null
• node cliqueFeatures[n]=
• edge cliqueFeatures[n]=

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/5349a19d62ff3e7a. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/stats/Distributions.java:32

 *
 *
 * @author Jeff Michels (jmichels@stanford.edu)
 */
public class Distributions {

  private Distributions() {
  }


  protected static <K> Set<K> getSetOfAllKeys(Distribution<K> d1, Distribution<K> d2) {
    if (d1.getNumberOfKeys() != d2.getNumberOfKeys()){
      throw new RuntimeException("Tried to compare two Distribution<K> objects but d1.numberOfKeys != d2.numberOfKeys");
    }

    Set<K> allKeys = Generics.newHashSet(d1.getCounter().keySet());
    allKeys.addAll(d2.getCounter().keySet());
    if (allKeys.size() > d1.getNumberOfKeys()){
      throw new RuntimeException("Tried to compare two Distribution<K> objects but d1.counter intersect d2.counter > numberOfKeys");
    }
    return allKeys;
  }

  /**
   * Returns a double between 0 and 1 representing the overlap of d1 and d2.
   * Equals 0 if there is no overlap, equals 1 iff d1==d2
   */
  public static <K> double overlap(Distribution<K> d1, Distribution<K> d2) {
    Set<K> allKeys = getSetOfAllKeys(d1, d2);

    double result = 0.0;
    double remainingMass1 = 1.0;
    double remainingMass2 = 1.0;

    for (K key : allKeys){
      double p1 = d1.probabilityOf(key);
      double p2 = d2.probabilityOf(key);

View on GitHub (pinned to 1b7edd19c4)