stanfordnlp/CoreNLP · error · IllegalArgumentException

This evaluator only works for the ShiftReduceParser

Error message

This evaluator only works for the ShiftReduceParser

What it means

TransitionTypeEval is a ParserQueryEval that counts which transitions the parser applied. It only understands ShiftReduceParserQuery objects; when handed any other parser's query implementation it cannot extract the transition sequence and throws IllegalArgumentException.

Solutions

  1. Use this evaluator only with ShiftReduceParser (remove it from the evaluator list for other parsers)
  2. Guard evaluation code: add the evaluator only when the parser is a ShiftReduceParser
  3. Choose an evaluator appropriate to the parser being tested

Example fix

// before
options.evals.add("transitionType"); // applied to every parser

// after
if (parser instanceof ShiftReduceParser) {
  options.evals.add("transitionType");
}
Defensive patterns

Strategy: type-guard

Validate before calling

if (!(query instanceof ShiftReduceParserQuery)) {
  return; // skip transitionType eval for this parser
}

Type guard

boolean supportsTransitionEval(ParserQuery q) {
  return q instanceof ShiftReduceParserQuery;
}

Try / catch

try {
  eval.evaluate(query, gold, pw);
} catch (IllegalArgumentException e) {
  log.warning("Skipping TransitionTypeEval: " + e.getMessage());
}

Prevention

When it happens

Trigger: Registering/running the transitionType evaluator during evaluation of a non-shift-reduce parser (e.g. an LRParser or other ParserQuery subclass), so evaluate() receives a ParserQuery that isn't ShiftReduceParserQuery.

Common situations: A shared evaluation config listing evaluator classes applied to multiple parsers; switching parsers in a test harness without updating the evaluators list.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/d280b0b14093ea7b. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/parser/shiftreduce/TransitionTypeEval.java:23

import edu.stanford.nlp.stats.Counters;
import edu.stanford.nlp.stats.IntCounter;
import edu.stanford.nlp.parser.common.ParserQuery;
import edu.stanford.nlp.parser.metrics.ParserQueryEval;
import edu.stanford.nlp.trees.Tree;

/**
 * Tally and output the number of each type of transition used.
 * Useful for cases where you are adding a new transition type and
 * want to make sure it is actually firing
 */
public class TransitionTypeEval implements ParserQueryEval {
  private IntCounter<Class<? extends Transition>> transitionCounts = new IntCounter<>();

  @Override
  public void evaluate(ParserQuery query, Tree gold, PrintWriter pw) {
    if (!(query instanceof ShiftReduceParserQuery)) {
      throw new IllegalArgumentException("This evaluator only works for the ShiftReduceParser");
    }

    ShiftReduceParserQuery srquery = (ShiftReduceParserQuery) query;
    List<Transition> transitions = srquery.getBestTransitionSequence();

    for (Transition t : transitions) {
      transitionCounts.incrementCount(t.getClass());
    }
  }

  @Override
  public void display(boolean verbose, PrintWriter pw) {
    pw.println("Shift-Reduce transition type frequency");
    List<Class<? extends Transition>> sorted = Counters.toSortedList(transitionCounts);
    for (Class<? extends Transition> t : sorted) {
      String className = ShiftReduceUtils.transitionShortName(t);
      pw.println("  " + className + ": " + transitionCounts.getCount(t));
    }

View on GitHub (pinned to 1b7edd19c4)