{"record":{"id":"30e585150dd4bf02","repo":"stanfordnlp/CoreNLP","slug":"conll-u-file-and-tree-file-are-not-aligned","errorCode":null,"errorMessage":"CoNLL-U file and tree file are not aligned. \n","messagePattern":"CoNLL-U file and tree file are not aligned\\. \n","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/trees/ud/UniversalDependenciesFeatureAnnotator.java","lineNumber":491,"sourceCode":"    CoNLLUDocumentReader depReader = new CoNLLUDocumentReader();\n    CoNLLUDocumentWriter depWriter = new CoNLLUDocumentWriter();\n    Iterator<Pair<SemanticGraph, SemanticGraph>> it = depReader.getIterator(r);\n\n    Iterator<Tree> treeIt = treebankIterator(treeFile);\n\n    while (it.hasNext()) {\n      SemanticGraph sg = it.next().first();\n      Tree t = treeIt.next();\n\n      if (t == null || t.yield().size() != sg.size()) {\n\n        StringBuilder sentenceSb = new StringBuilder();\n        for (IndexedWord word : sg.vertexListSorted()) {\n          sentenceSb.append(word.get(CoreAnnotations.TextAnnotation.class));\n          sentenceSb.append(' ');\n        }\n\n        throw new RuntimeException(\"CoNLL-U file and tree file are not aligned. \\n\"\n                + \"Sentence: \" + sentenceSb + '\\n'\n                + \"Tree: \" + ((t == null) ? \"null\" : t.pennString()));\n      }\n\n      featureAnnotator.addFeatures(sg, t, true, addUPOS);\n\n      System.out.print(depWriter.printSemanticGraph(sg, null, !escapeParens));\n    }\n  }\n\n}\n\n","sourceCodeStart":473,"sourceCodeEnd":504,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/trees/ud/UniversalDependenciesFeatureAnnotator.java#L473-L504","documentation":"UniversalDependenciesFeatureAnnotator throws this when the CoNLL-U sentence graph and the corresponding tree from the parallel tree file cannot be aligned — typically the number of tokens in the semantic graph does not match the leaves of the tree. It includes the raw sentence text from the graph and the Penn string of the tree to help diagnose the mismatch.","triggerScenarios":"During main(), after reading an sg and a tree t from two parallel files, a length/content check on sg.vertexListSorted() vs t's leaves fails (including t == null), so the RuntimeException with both renderings is thrown.","commonSituations":"Tree file and CoNLL-U file from different versions of a corpus; tokenization differences (hyphens, multi-word tokens split differently); blank-line misalignment shifting all subsequent sentences; a null tree because the tree reader hit EOF or a parse error earlier in the file.","solutions":["Compare the printed sentence and tree in the message: count tokens vs tree leaves and find where they diverge.","Regenerate both files from the same corpus version so tokenization matches.","Check for blank-line misalignment near the failing sentence (message prints the offending sentence, so check surrounding blocks).","Verify the tree file actually contains a tree for every CoNLL-U sentence (no truncation causing null t)."],"exampleFix":"// before\nthrow new RuntimeException(\"CoNLL-U file and tree file are not aligned. \\n\" ...);\n// after (caller-side guard)\nif (t == null || t.getLeaves().length != sg.vertexListSorted().size()) {\n  throw new IOException(\"Misalignment at sent_id=\" + sg graphIdx + \": check blank lines in CoNLL-U/tree files\");\n}","handlingStrategy":"validation","validationCode":"int graphTokens = sg.vertexListSorted().size();\nint treeLeaves = (t == null) ? -1 : t.getLeaves().length;\nif (graphTokens != treeLeaves) throw new IllegalStateException(\"Token/leaf mismatch: \" + graphTokens + \" vs \" + treeLeaves + \" — check file alignment\");","typeGuard":null,"tryCatchPattern":"try { annotator.process(sg, t); } catch (RuntimeException e) { if (e.getMessage().startsWith(\"CoNLL-U file and tree file are not aligned\")) { logAlignmentDiagnostic(e.getMessage()); } else { throw e; } }","preventionTips":["Regenerate tree and CoNLL-U files together so tokenization matches.","Watch for multi-word token splits that change token counts.","Diff the failing sentence (printed in the message) against the corpus source."],"tags":["alignment","conllu","corpus","tokenization"],"backgroundTag":"schema-validation-failed","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}