{"record":{"id":"675601a0512804bf","repo":"stanfordnlp/CoreNLP","slug":"error-numberofkeys-d-must-be-size-of-counter","errorCode":null,"errorMessage":"ERROR: numberOfKeys %d must be > size of counter %d!","messagePattern":"ERROR: numberOfKeys (.+?) must be > size of counter (.+?)!","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/stats/Distribution.java","lineNumber":455,"sourceCode":"    return countCounts;\n  }\n\n\n  // ----------------------------------------------------------------------------\n\n  /**\n   * Creates a Distribution from the given counter using Gale &amp; Sampsons'\n   * \"simple Good-Turing\" smoothing.\n   *\n   * @return a new simple Good-Turing smoothed Distribution.\n   */\n  public static <E> Distribution<E> simpleGoodTuring(Counter<E> counter, int numberOfKeys) {\n\n    // check arguments\n    validateCounter(counter);\n    int numUnseen = numberOfKeys - counter.size();\n    if (numUnseen < 1)\n      throw new IllegalArgumentException(String.format(\"ERROR: numberOfKeys %d must be > size of counter %d!\", numberOfKeys, counter.size()));\n\n    // do smoothing\n    int[][] cc = countCounts2IntArrays(collectCountCounts(counter));\n    int[] r = cc[0];                    // counts\n    int[] n = cc[1];                    // counts of counts\n    SimpleGoodTuring sgt = new SimpleGoodTuring(r, n);\n\n    // collate results\n    Counter<Integer> probsByCount = new ClassicCounter<>();\n    double[] probs = sgt.getProbabilities();\n    for (int i = 0; i < probs.length; i++) {\n      probsByCount.setCount(r[i], probs[i]);\n    }\n\n    // make smoothed distribution\n    Distribution<E> dist = new Distribution<>();\n    dist.counter = new ClassicCounter<>();\n    for (Map.Entry<E, Double> entry : counter.entrySet()) {","sourceCodeStart":437,"sourceCodeEnd":473,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/stats/Distribution.java#L437-L473","documentation":"simpleGoodTuring() builds a smoothed distribution over numberOfKeys total possible outcomes, where only counter.size() outcomes have been observed. It requires at least one unobserved key so that smoothed probability mass can be reserved for unseen items. If numberOfKeys does not exceed the counter's size, the smoothing is ill-defined and an IllegalArgumentException is thrown.","triggerScenarios":"Calling Distribution.simpleGoodTuring(counter, numberOfKeys) with numberOfKeys <= counter.size(), e.g. passing the counter's size() as numberOfKeys, or forgetting that numberOfKeys must count the full vocabulary including unseen symbols.","commonSituations":"Language-model smoothing where the developer mistakenly passes the observed vocabulary size instead of the total type count (observed + unseen word types); off-by-one when numberOfKeys equals counter.size().","solutions":["Pass a numberOfKeys strictly greater than counter.size(), typically the total vocabulary size including unseen types","Check with counter.size() before calling: if numberOfKeys <= counter.size(), enlarge numberOfKeys or use a different smoothing method","If there truly are no unseen items, use Distribution.getDistribution(counter) or another smoother that does not reserve unseen mass"],"exampleFix":"// before\nDistribution<String> d = Distribution.simpleGoodTuring(counter, counter.size());\n// after\nint totalVocab = observedWords + unseenWordTypes;\nDistribution<String> d = Distribution.simpleGoodTuring(counter, totalVocab); // totalVocab > counter.size()","handlingStrategy":"validation","validationCode":"if (numberOfKeys <= counter.size()) {\n  throw new IllegalArgumentException(\"numberOfKeys (\" + numberOfKeys + \") must exceed counter size (\" + counter.size() + \")\");\n}\nDistribution<String> d = Distribution.simpleGoodTuring(counter, numberOfKeys);","typeGuard":null,"tryCatchPattern":"try {\n  Distribution<String> d = Distribution.simpleGoodTuring(counter, numberOfKeys);\n} catch (IllegalArgumentException e) {\n  // fall back to unsmoothed distribution or fix numberOfKeys\n}","preventionTips":["Always pass the total universe size (observed + unseen types), never counter.size()","Assert numberOfKeys > counter.size() before calling","Unit-test smoothing with counters whose size equals an intentional off-by-one numberOfKeys"],"tags":["java","illegal-argument","smoothing","statistics"],"backgroundTag":"invalid-argument-value","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}