{"record":{"id":"d537e9fd148d51d5","repo":"stanfordnlp/CoreNLP","slug":"mismatch-between-number-of-lemmas-and-number-of-in","errorCode":null,"errorMessage":"Mismatch between number of lemmas and number of input lines.","messagePattern":"Mismatch between number of lemmas and number of input lines\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/international/morph/AddMorphoAnnotations.java","lineNumber":176,"sourceCode":"    YieldIterator morphIter = new YieldIterator(parsedArgs[0], isMorphTreeFile);\n    YieldIterator lemmaIter = new YieldIterator(parsedArgs[1], false);\n    \n    final Pattern pParenStripper = Pattern.compile(\"[\\\\(\\\\)]\");\n        \n    try {\n      BufferedReader brIn = new BufferedReader(new InputStreamReader(System.in, encoding));\n      TreeReaderFactory trf = new ArabicTreeReaderFactory.ArabicRawTreeReaderFactory(true);\n\n      int nTrees = 0;\n      for(String line; (line = brIn.readLine()) != null; ++nTrees) {\n        Tree tree = trf.newTreeReader(new StringReader(line)).readTree();\n        List<Tree> leaves = tree.getLeaves();\n        if(!morphIter.hasNext()) {\n          throw new RuntimeException(\"Mismatch between number of morpho analyses and number of input lines.\");\n        }\n        List<String> morphTags = morphIter.next();\n        if (!lemmaIter.hasNext()) {\n          throw new RuntimeException(\"Mismatch between number of lemmas and number of input lines.\");\n        }\n        List<String> lemmas = lemmaIter.next();\n         \n        // Sanity checks\n        assert morphTags.size() == lemmas.size();\n        assert lemmas.size() == leaves.size();\n        \n        for(int i = 0; i < leaves.size(); ++i) {\n          String morphTag = morphTags.get(i);\n          if (pParenStripper.matcher(morphTag).find()) {\n            morphTag = pParenStripper.matcher(morphTag).replaceAll(\"\");\n          }\n          String newLeaf = String.format(\"%s%s%s%s%s\", leaves.get(i).value(),\n              MorphoFeatureSpecification.MORPHO_MARK,\n              lemmas.get(i),\n              MorphoFeatureSpecification.LEMMA_MARK,\n              morphTag);\n          leaves.get(i).setValue(newLeaf);","sourceCodeStart":158,"sourceCodeEnd":194,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/international/morph/AddMorphoAnnotations.java#L158-L194","documentation":"Same loop as error 337: after consuming the morpho analyses, the code verifies a parallel lemma iterator still has an entry for each tree line and throws this RuntimeException when the lemma file has fewer entries than the tree input.","triggerScenarios":"Running AddMorphoAnnotations where the lemma file is shorter than the tree file, so after morphTags are consumed the lemma iterator is exhausted mid-loop.","commonSituations":"Truncated lemma file, lemma and morpho files generated from different corpus versions, accidental line filtering.","solutions":["Ensure the lemma file has exactly one entry per input tree line","Regenerate both lemma and morpho files from the same treebank version","Verify no preprocessing dropped lines from only the lemma file"],"exampleFix":"// before\nwc -l lemmas.txt  # 990\nwc -l trees.txt   # 1000\n// after\nwc -l lemmas.txt  # 1000 (regenerate to match)","handlingStrategy":"validation","validationCode":"if (nTreeLines != nLemmaLines) throw new IllegalStateException(\"trees=\" + nTreeLines + \" lemmas=\" + nLemmaLines);","typeGuard":null,"tryCatchPattern":"try { process(); } catch (RuntimeException e) { log.error(\"Lemma/tree count mismatch: \" + e.getMessage()); System.exit(2); }","preventionTips":["Verify all parallel annotation files have equal line counts","Regenerate annotations together from one source snapshot","Use a shared preprocessing step so line filtering is consistent"],"tags":["corpus","annotation","data-mismatch"],"backgroundTag":"empty-result-set","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}