{"record":{"id":"8cf6792dcf617944","repo":"stanfordnlp/CoreNLP","slug":"attachmentscore-cannot-be-used-when-count-gold-dep","errorCode":null,"errorMessage":"AttachmentScore cannot be used when count(gold deps:%d) != count(system deps:%d)","messagePattern":"AttachmentScore cannot be used when count\\(gold deps:(.+?)\\) != count\\(system deps:(.+?)\\)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/trees/DependencyScoring.java","lineNumber":351,"sourceCode":"      this.parserCnt = parserCnt;\n      this.goldCnt = goldCnt;\n      this.parserUnlabeledCnt = parserUnlabeledCnt;\n      this.goldUnlabeledCnt = goldUnlabeledCnt;\n      this.correctAttachment = correctAttachment;\n      this.correctUnlabeledAttachment = correctUnlabeledAttachment;\n      this.labelCnt = labelCnt;\n      this.labelCorrect = labelCorrect;\n      this.unlabeledErrorCounts = new ClassicCounter<>(unlabeledErrorCounts);\n      this.labeledErrorCounts = new ClassicCounter<>(labeledErrorCounts);\n    }\n\n    public String toString() {\n      return toStringFScore(false, false);\n    }\n\n    public String toStringAttachmentScore(boolean json) {\n       if (parserCnt != goldCnt) {\n         throw new RuntimeException(\n           String.format(\"AttachmentScore cannot be used when count(gold deps:%d) != count(system deps:%d)\", parserCnt, goldCnt));\n       }\n       double las = correctAttachment/(double)goldCnt;\n       double uas = correctUnlabeledAttachment/(double)goldCnt;\n       StringBuilder sbuild = new StringBuilder();\n\n       if (json) {\n         sbuild.append(\"{\");\n         sbuild.append(String.format(\"'LAS' : %.3f, \", las));\n         sbuild.append(String.format(\"'UAS' : %.3f, \", uas));\n         sbuild.append(\"}\");\n       } else {\n         sbuild.append(String.format(\"|| Labeled Attachment Score   ||\"));\n         sbuild.append(String.format(\" %.3f (%d/%d) ||\\n\", las, correctAttachment, goldCnt));\n\n         sbuild.append(String.format(\"|| Unlabeled Attachment Score ||\"));\n         sbuild.append(String.format(\" %.3f (%d/%d) ||\\n\", uas, correctUnlabeledAttachment, goldCnt));\n       }","sourceCodeStart":333,"sourceCodeEnd":369,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/trees/DependencyScoring.java#L333-L369","documentation":"Attachment scores are computed as correct/goldCnt, which is only meaningful when the gold and system dependency files contain the same number of dependencies. toStringAttachmentScore enforces this invariant (parserCnt == goldCnt) and throws otherwise, since divergent counts mean sentences failed to parse or were misaligned.","triggerScenarios":"Calling toStringAttachmentScore(true/false) after scoring files where the system output has a different dependency count than the gold file — typically because some sentences produced no parse or files were truncated/mismatched.","commonSituations":"Parser skipped/failed on some sentences producing fewer dependencies; accidentally scoring two different files or different sentence splits; counting dropped punctuation/functional tokens differently between gold and system outputs.","solutions":["Re-run the parser on all gold sentences so system output has the same dependency count, then rescore.","Verify you are scoring the matched gold/system file pair with identical tokenization and sentence order.","If a few sentences legitimately lack parses, repair them (e.g. attach to root) so counts match, or report coverage separately and only compute attachment on matched subsets via a scorer mode that supports it."],"exampleFix":"// before\nscorer.toStringAttachmentScore(false); // goldCnt=1000, parserCnt=985 -> throws\n// after: reparse missing 15 sentences or check counts first\nif (parserCnt != goldCnt) { fixMissingParses(); }\nString s = scorer.toStringAttachmentScore(false);","handlingStrategy":"validation","validationCode":"if (scorer.parserCnt != scorer.goldCnt) { throw new IllegalStateException(\"gold/system dependency counts differ: \" + scorer.goldCnt + \" vs \" + scorer.parserCnt); }","typeGuard":null,"tryCatchPattern":"try { String s = scorer.toStringAttachmentScore(false); } catch (RuntimeException e) { /* rescore after re-parsing missing sentences */ }","preventionTips":["Check gold/system dependency counts before requesting attachment scores","Re-parse any sentence the parser failed on so counts align","Keep tokenization identical between gold and system files"],"tags":["java","nlp","evaluation","internal-invariant-violation"],"backgroundTag":"internal-invariant-violation","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}