stanfordnlp/CoreNLP · error · RuntimeException

AttachmentScore cannot be used when count

Error message

AttachmentScore cannot be used when count(gold deps:%d) != count(system deps:%d)

What it means

Attachment scores are computed as correct/goldCnt, which is only meaningful when the gold and system dependency files contain the same number of dependencies. toStringAttachmentScore enforces this invariant (parserCnt == goldCnt) and throws otherwise, since divergent counts mean sentences failed to parse or were misaligned.

Solutions

Re-run the parser on all gold sentences so system output has the same dependency count, then rescore.
Verify you are scoring the matched gold/system file pair with identical tokenization and sentence order.
If a few sentences legitimately lack parses, repair them (e.g. attach to root) so counts match, or report coverage separately and only compute attachment on matched subsets via a scorer mode that supports it.

Example fix

// before
scorer.toStringAttachmentScore(false); // goldCnt=1000, parserCnt=985 -> throws
// after: reparse missing 15 sentences or check counts first
if (parserCnt != goldCnt) { fixMissingParses(); }
String s = scorer.toStringAttachmentScore(false);

Defensive patterns

Strategy: validation

Validate before calling

if (scorer.parserCnt != scorer.goldCnt) { throw new IllegalStateException("gold/system dependency counts differ: " + scorer.goldCnt + " vs " + scorer.parserCnt); }

Try / catch

try { String s = scorer.toStringAttachmentScore(false); } catch (RuntimeException e) { /* rescore after re-parsing missing sentences */ }

Prevention

Check gold/system dependency counts before requesting attachment scores
Re-parse any sentence the parser failed on so counts align
Keep tokenization identical between gold and system files

When it happens

Trigger: Calling toStringAttachmentScore(true/false) after scoring files where the system output has a different dependency count than the gold file — typically because some sentences produced no parse or files were truncated/mismatched.

Common situations: Parser skipped/failed on some sentences producing fewer dependencies; accidentally scoring two different files or different sentence splits; counting dropped punctuation/functional tokens differently between gold and system outputs.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/8cf6792dcf617944. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/trees/DependencyScoring.java:351

      this.parserCnt = parserCnt;
      this.goldCnt = goldCnt;
      this.parserUnlabeledCnt = parserUnlabeledCnt;
      this.goldUnlabeledCnt = goldUnlabeledCnt;
      this.correctAttachment = correctAttachment;
      this.correctUnlabeledAttachment = correctUnlabeledAttachment;
      this.labelCnt = labelCnt;
      this.labelCorrect = labelCorrect;
      this.unlabeledErrorCounts = new ClassicCounter<>(unlabeledErrorCounts);
      this.labeledErrorCounts = new ClassicCounter<>(labeledErrorCounts);
    }

    public String toString() {
      return toStringFScore(false, false);
    }

    public String toStringAttachmentScore(boolean json) {
       if (parserCnt != goldCnt) {
         throw new RuntimeException(
           String.format("AttachmentScore cannot be used when count(gold deps:%d) != count(system deps:%d)", parserCnt, goldCnt));
       }
       double las = correctAttachment/(double)goldCnt;
       double uas = correctUnlabeledAttachment/(double)goldCnt;
       StringBuilder sbuild = new StringBuilder();

       if (json) {
         sbuild.append("{");
         sbuild.append(String.format("'LAS' : %.3f, ", las));
         sbuild.append(String.format("'UAS' : %.3f, ", uas));
         sbuild.append("}");
       } else {
         sbuild.append(String.format("|| Labeled Attachment Score   ||"));
         sbuild.append(String.format(" %.3f (%d/%d) ||\n", las, correctAttachment, goldCnt));

         sbuild.append(String.format("|| Unlabeled Attachment Score ||"));
         sbuild.append(String.format(" %.3f (%d/%d) ||\n", uas, correctUnlabeledAttachment, goldCnt));
       }

View on GitHub (pinned to 1b7edd19c4)