{"record":{"id":"ce8c9ff624e71dde","repo":"stanfordnlp/CoreNLP","slug":"originalwords-and-sentence-of-different-sizes-or","errorCode":null,"errorMessage":"originalWords and sentence of different sizes: <originalSentence.size> vs. <leaves.size>\n Orig: <SentenceUtils.listToString(originalSentence)>\n Pars: <SentenceUtils.listToString(leaves)>","messagePattern":"originalWords and sentence of different sizes: <originalSentence\\.size> vs\\. <leaves\\.size>\n Orig: <SentenceUtils\\.listToString\\(originalSentence\\)>\n Pars: <SentenceUtils\\.listToString\\(leaves\\)>","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/parser/metrics/ExternalParserQuery.java","lineNumber":102,"sourceCode":"\n  @Override\n  public Tree getBestFactoredParse() {\n    return null;\n  }\n\n  @Override\n  public List<ScoredObject<Tree>> getBestPCFGParses() {\n    return results;\n  }\n\n  @Override\n  public void restoreOriginalWords(Tree tree) {\n    if (originalSentence == null || tree == null) {\n      return;\n    }\n    List<Tree> leaves = tree.getLeaves();\n    if (leaves.size() != originalSentence.size()) {\n      throw new IllegalStateException(\"originalWords and sentence of different sizes: \" + originalSentence.size() + \" vs. \" + leaves.size() +\n                                      \"\\n Orig: \" + SentenceUtils.listToString(originalSentence) +\n                                      \"\\n Pars: \" + SentenceUtils.listToString(leaves));\n    }\n    Iterator<Tree> leafIterator = leaves.iterator();\n    for (HasWord word : originalSentence) {\n      Tree leaf = leafIterator.next();\n      if (!(word instanceof Label)) {\n        continue;\n      }\n      leaf.setLabel((Label) word);\n    }\n  }\n\n  @Override\n  public boolean hasFactoredParse() {\n    return false;\n  }\n","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/parser/metrics/ExternalParserQuery.java#L84-L120","documentation":"restoreOriginalWords() copies the original (pre-preprocessing) word list back onto the leaf nodes of a parse tree. It throws IllegalStateException when the number of tree leaves differs from the number of original words, meaning the tree was modified (tokens added/removed during parsing) and the mapping back to original words is impossible. This is a safety check so output trees line up with the input sentence.","triggerScenarios":"Calling restoreOriginalWords(tree) with a tree whose leaf count != originalSentence.size(), typically after the parser inserted/deleted tokens (e.g. added -LRB-/missing punctuation or a dummy root).","commonSituations":"Parsing with a treebank language pack that adds final punctuation or normalizes tokens; passing a tree from a different sentence than the one stored; post-processing the tree (pruning/pruning nodes) before restoring words.","solutions":["Ensure the tree passed in comes from parsing exactly the sentence stored in the query (originalSentence)","Disable token normalization/insertion options that change token count (e.g. -makeCopulaHead, -changePunctGuillemets, addFinalPeriod options) or be aware they change leaf count","If the tree was post-processed, restore original words before modifying leaves","Log both the original sentence and the leaves (as the exception does) to find where counts diverge"],"exampleFix":"// before\nparserQuery.parse(preprocessedSentence);\nTree t = parserQuery.getBestParse();\nquery.restoreOriginalWords(t); // leaf count mismatch\n// after\nparserQuery.parse(originalSentence); // parse the original sentence so leaves match\nTree t = parserQuery.getBestParse();\nquery.restoreOriginalWords(t);","handlingStrategy":"validation","validationCode":"List<Tree> leaves = tree.getLeaves();\nif (leaves.size() != originalSentence.size()) {\n  throw new IllegalArgumentException(\"leaf/original mismatch: \" + leaves.size() + \" vs \" + originalSentence.size());\n}\nquery.restoreOriginalWords(tree);","typeGuard":null,"tryCatchPattern":"try {\n  query.restoreOriginalWords(tree);\n} catch (IllegalStateException e) {\n  log.warning(\"Tree leaves do not match original words; skipping word restoration\");\n}","preventionTips":["Parse exactly the sentence you intend to restore words onto","Disable token-adding/normalizing parser options or account for them","Restore words before any post-processing that alters leaves"],"tags":["java","illegal-state","nlp","tree"],"backgroundTag":"internal-invariant-violation","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}