{"record":{"id":"d280b0b14093ea7b","repo":"stanfordnlp/CoreNLP","slug":"this-evaluator-only-works-for-the-shiftreduceparse","errorCode":null,"errorMessage":"This evaluator only works for the ShiftReduceParser","messagePattern":"This evaluator only works for the ShiftReduceParser","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/parser/shiftreduce/TransitionTypeEval.java","lineNumber":23,"sourceCode":"\nimport edu.stanford.nlp.stats.Counters;\nimport edu.stanford.nlp.stats.IntCounter;\nimport edu.stanford.nlp.parser.common.ParserQuery;\nimport edu.stanford.nlp.parser.metrics.ParserQueryEval;\nimport edu.stanford.nlp.trees.Tree;\n\n/**\n * Tally and output the number of each type of transition used.\n * Useful for cases where you are adding a new transition type and\n * want to make sure it is actually firing\n */\npublic class TransitionTypeEval implements ParserQueryEval {\n  private IntCounter<Class<? extends Transition>> transitionCounts = new IntCounter<>();\n\n  @Override\n  public void evaluate(ParserQuery query, Tree gold, PrintWriter pw) {\n    if (!(query instanceof ShiftReduceParserQuery)) {\n      throw new IllegalArgumentException(\"This evaluator only works for the ShiftReduceParser\");\n    }\n\n    ShiftReduceParserQuery srquery = (ShiftReduceParserQuery) query;\n    List<Transition> transitions = srquery.getBestTransitionSequence();\n\n    for (Transition t : transitions) {\n      transitionCounts.incrementCount(t.getClass());\n    }\n  }\n\n  @Override\n  public void display(boolean verbose, PrintWriter pw) {\n    pw.println(\"Shift-Reduce transition type frequency\");\n    List<Class<? extends Transition>> sorted = Counters.toSortedList(transitionCounts);\n    for (Class<? extends Transition> t : sorted) {\n      String className = ShiftReduceUtils.transitionShortName(t);\n      pw.println(\"  \" + className + \": \" + transitionCounts.getCount(t));\n    }","sourceCodeStart":5,"sourceCodeEnd":41,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/parser/shiftreduce/TransitionTypeEval.java#L5-L41","documentation":"TransitionTypeEval is a ParserQueryEval that counts which transitions the parser applied. It only understands ShiftReduceParserQuery objects; when handed any other parser's query implementation it cannot extract the transition sequence and throws IllegalArgumentException.","triggerScenarios":"Registering/running the transitionType evaluator during evaluation of a non-shift-reduce parser (e.g. an LRParser or other ParserQuery subclass), so evaluate() receives a ParserQuery that isn't ShiftReduceParserQuery.","commonSituations":"A shared evaluation config listing evaluator classes applied to multiple parsers; switching parsers in a test harness without updating the evaluators list.","solutions":["Use this evaluator only with ShiftReduceParser (remove it from the evaluator list for other parsers)","Guard evaluation code: add the evaluator only when the parser is a ShiftReduceParser","Choose an evaluator appropriate to the parser being tested"],"exampleFix":"// before\noptions.evals.add(\"transitionType\"); // applied to every parser\n\n// after\nif (parser instanceof ShiftReduceParser) {\n  options.evals.add(\"transitionType\");\n}","handlingStrategy":"type-guard","validationCode":"if (!(query instanceof ShiftReduceParserQuery)) {\n  return; // skip transitionType eval for this parser\n}","typeGuard":"boolean supportsTransitionEval(ParserQuery q) {\n  return q instanceof ShiftReduceParserQuery;\n}","tryCatchPattern":"try {\n  eval.evaluate(query, gold, pw);\n} catch (IllegalArgumentException e) {\n  log.warning(\"Skipping TransitionTypeEval: \" + e.getMessage());\n}","preventionTips":["Register shift-reduce-specific evaluators only for ShiftReduceParser runs","Filter evaluator lists by parser type","Add instanceof checks in generic eval harnesses"],"tags":["parser","evaluation","type-mismatch","api-usage"],"backgroundTag":"incompatible-source-type","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}