stanfordnlp/CoreNLP · error · IllegalArgumentException

ERROR: numberOfKeys must be > size of counter !

Error message

ERROR: numberOfKeys %d must be > size of counter %d!

What it means

simpleGoodTuring() builds a smoothed distribution over numberOfKeys total possible outcomes, where only counter.size() outcomes have been observed. It requires at least one unobserved key so that smoothed probability mass can be reserved for unseen items. If numberOfKeys does not exceed the counter's size, the smoothing is ill-defined and an IllegalArgumentException is thrown.

Solutions

1. Pass a numberOfKeys strictly greater than counter.size(), typically the total vocabulary size including unseen types
2. Check with counter.size() before calling: if numberOfKeys <= counter.size(), enlarge numberOfKeys or use a different smoothing method
3. If there truly are no unseen items, use Distribution.getDistribution(counter) or another smoother that does not reserve unseen mass

Example fix

// before
Distribution<String> d = Distribution.simpleGoodTuring(counter, counter.size());
// after
int totalVocab = observedWords + unseenWordTypes;
Distribution<String> d = Distribution.simpleGoodTuring(counter, totalVocab); // totalVocab > counter.size()

if (numberOfKeys <= counter.size()) {
  throw new IllegalArgumentException("numberOfKeys (" + numberOfKeys + ") must exceed counter size (" + counter.size() + ")");
}
Distribution<String> d = Distribution.simpleGoodTuring(counter, numberOfKeys);

try {
  Distribution<String> d = Distribution.simpleGoodTuring(counter, numberOfKeys);
} catch (IllegalArgumentException e) {
  // fall back to unsmoothed distribution or fix numberOfKeys
}

When it happens

Trigger: Calling Distribution.simpleGoodTuring(counter, numberOfKeys) with numberOfKeys <= counter.size(), e.g. passing the counter's size() as numberOfKeys, or forgetting that numberOfKeys must count the full vocabulary including unseen symbols.

Common situations: Language-model smoothing where the developer mistakenly passes the observed vocabulary size instead of the total type count (observed + unseen word types); off-by-one when numberOfKeys equals counter.size().

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Bad arguments: " + x + " and " + lambda
• Cannot make normalized counter with Dynamic prior.
• Cannot put a child trie with no keys
• CoreMap must have either a Calendar or DocDate annotation
• Could not compute span from tokens!

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/675601a0512804bf. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/stats/Distribution.java:455

    return countCounts;
  }


  // ----------------------------------------------------------------------------

  /**
   * Creates a Distribution from the given counter using Gale & Sampsons'
   * "simple Good-Turing" smoothing.
   *
   * @return a new simple Good-Turing smoothed Distribution.
   */
  public static <E> Distribution<E> simpleGoodTuring(Counter<E> counter, int numberOfKeys) {

    // check arguments
    validateCounter(counter);
    int numUnseen = numberOfKeys - counter.size();
    if (numUnseen < 1)
      throw new IllegalArgumentException(String.format("ERROR: numberOfKeys %d must be > size of counter %d!", numberOfKeys, counter.size()));

    // do smoothing
    int[][] cc = countCounts2IntArrays(collectCountCounts(counter));
    int[] r = cc[0];                    // counts
    int[] n = cc[1];                    // counts of counts
    SimpleGoodTuring sgt = new SimpleGoodTuring(r, n);

    // collate results
    Counter<Integer> probsByCount = new ClassicCounter<>();
    double[] probs = sgt.getProbabilities();
    for (int i = 0; i < probs.length; i++) {
      probsByCount.setCount(r[i], probs[i]);
    }

    // make smoothed distribution
    Distribution<E> dist = new Distribution<>();
    dist.counter = new ClassicCounter<>();
    for (Map.Entry<E, Double> entry : counter.entrySet()) {

View on GitHub (pinned to 1b7edd19c4)