stanfordnlp/CoreNLP · error · IllegalStateException

originalWords and sentence of different sizes: <originalSent

Error message

originalWords and sentence of different sizes: <originalSentence.size> vs. <leaves.size>
 Orig: <SentenceUtils.listToString(originalSentence)>
 Pars: <SentenceUtils.listToString(leaves)>

What it means

restoreOriginalWords() copies the original (pre-preprocessing) word list back onto the leaf nodes of a parse tree. It throws IllegalStateException when the number of tree leaves differs from the number of original words, meaning the tree was modified (tokens added/removed during parsing) and the mapping back to original words is impossible. This is a safety check so output trees line up with the input sentence.

Solutions

  1. Ensure the tree passed in comes from parsing exactly the sentence stored in the query (originalSentence)
  2. Disable token normalization/insertion options that change token count (e.g. -makeCopulaHead, -changePunctGuillemets, addFinalPeriod options) or be aware they change leaf count
  3. If the tree was post-processed, restore original words before modifying leaves
  4. Log both the original sentence and the leaves (as the exception does) to find where counts diverge

Example fix

// before
parserQuery.parse(preprocessedSentence);
Tree t = parserQuery.getBestParse();
query.restoreOriginalWords(t); // leaf count mismatch
// after
parserQuery.parse(originalSentence); // parse the original sentence so leaves match
Tree t = parserQuery.getBestParse();
query.restoreOriginalWords(t);
Defensive patterns

Strategy: validation

Validate before calling

List<Tree> leaves = tree.getLeaves();
if (leaves.size() != originalSentence.size()) {
  throw new IllegalArgumentException("leaf/original mismatch: " + leaves.size() + " vs " + originalSentence.size());
}
query.restoreOriginalWords(tree);

Try / catch

try {
  query.restoreOriginalWords(tree);
} catch (IllegalStateException e) {
  log.warning("Tree leaves do not match original words; skipping word restoration");
}

Prevention

When it happens

Trigger: Calling restoreOriginalWords(tree) with a tree whose leaf count != originalSentence.size(), typically after the parser inserted/deleted tokens (e.g. added -LRB-/missing punctuation or a dummy root).

Common situations: Parsing with a treebank language pack that adds final punctuation or normalizes tokens; passing a tree from a different sentence than the one stored; post-processing the tree (pruning/pruning nodes) before restoring words.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/ce8c9ff624e71dde. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/parser/metrics/ExternalParserQuery.java:102

  @Override
  public Tree getBestFactoredParse() {
    return null;
  }

  @Override
  public List<ScoredObject<Tree>> getBestPCFGParses() {
    return results;
  }

  @Override
  public void restoreOriginalWords(Tree tree) {
    if (originalSentence == null || tree == null) {
      return;
    }
    List<Tree> leaves = tree.getLeaves();
    if (leaves.size() != originalSentence.size()) {
      throw new IllegalStateException("originalWords and sentence of different sizes: " + originalSentence.size() + " vs. " + leaves.size() +
                                      "\n Orig: " + SentenceUtils.listToString(originalSentence) +
                                      "\n Pars: " + SentenceUtils.listToString(leaves));
    }
    Iterator<Tree> leafIterator = leaves.iterator();
    for (HasWord word : originalSentence) {
      Tree leaf = leafIterator.next();
      if (!(word instanceof Label)) {
        continue;
      }
      leaf.setLabel((Label) word);
    }
  }

  @Override
  public boolean hasFactoredParse() {
    return false;
  }

View on GitHub (pinned to 1b7edd19c4)