stanfordnlp/CoreNLP · error · java.lang.IllegalArgumentException

Sentence.toSentence: lengths differ

Error message

Sentence.toSentence: lengths differ

What it means

SentenceUtils.toTaggedList builds a TaggedWord list by pairing the lex (words) and tags lists element-by-element. If the two input lists have different sizes, the method throws IllegalArgumentException("Sentence.toSentence: lengths differ") because every word must have exactly one tag. This is an input-validation guard, not a library bug.

Solutions

1. Validate lex.size() == tags.size() before the call and fix whichever list is wrong.
2. Ensure both lists come from the same tokenization pass — don't filter or split one list independently of the other.
3. If lists may legitimately differ, zip to the min length yourself and construct TaggedWord pairs in a loop instead.
4. Log both sizes at the call site to identify which pipeline stage desynchronized the lists.

Example fix

// before
List<TaggedWord> sent = SentenceUtils.toTaggedList(lex, tags); // throws if sizes differ
// after
if (lex.size() != tags.size()) {
  throw new IllegalArgumentException("lex=" + lex.size() + " tags=" + tags.size());
}
List<TaggedWord> sent = SentenceUtils.toTaggedList(lex, tags);

if (lex == null || tags == null || lex.size() != tags.size()) {
  throw new IllegalArgumentException(
      "toTaggedList requires equal sizes: lex=" + (lex == null ? -1 : lex.size())
      + " tags=" + (tags == null ? -1 : tags.size()));
}
List<TaggedWord> sent = SentenceUtils.toTaggedList(lex, tags);

boolean isParallel(List<?> a, List<?> b) {
  return a != null && b != null && a.size() == b.size();
}

try {
  return SentenceUtils.toTaggedList(lex, tags);
} catch (IllegalArgumentException e) {
  if (!e.getMessage().contains("lengths differ")) throw e;
  log.error("lex.size=" + lex.size() + " tags.size=" + tags.size());
  int n = Math.min(lex.size(), tags.size());
  return SentenceUtils.toTaggedList(lex.subList(0, n), tags.subList(0, n));
}

When it happens

Trigger: Calling SentenceUtils.toTaggedList(lex, tags) (or the equivalent Sentence.toSentence overloads) with lex.size() != tags.size(), e.g. after a tokenization step dropped or added items on one side.

Common situations: Piping tokens from a tokenizer into a POS tagger where empty tokens were filtered from the word list but not the tag list; reading parallel word/tag data from a file with a ragged last line; off-by-one trimming of whitespace-only tokens.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Unknown clique: " + clique
• Unknown format
• Invalid arguments to
• CoreMap must have either a Calendar or DocDate annotation
• Unsupported subScoreType

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/c67505ad82fb2d8b. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ling/SentenceUtils.java:39

  private SentenceUtils() {} // static methods

  /**
   * Create an ArrayList as a list of {@code TaggedWord} from two
   * lists of {@code String}, one for the words, and the second for
   * the tags.
   *
   * @param lex  a list whose items are of type {@code String} and
   *             are the words
   * @param tags a list whose items are of type {@code String} and
   *             are the tags
   * @return The Sentence
   */
  public static ArrayList<TaggedWord> toTaggedList(List<String> lex, List<String> tags) {
    ArrayList<TaggedWord> sent = new ArrayList<>();
    int ls = lex.size();
    int ts = tags.size();
    if (ls != ts) {
      throw new IllegalArgumentException("Sentence.toSentence: lengths differ");
    }
    for (int i = 0; i < ls; i++) {
      sent.add(new TaggedWord(lex.get(i), tags.get(i)));
    }
    return sent;
  }

  /**
   * Create an ArrayList as a list of {@code Word} from a
   * list of {@code String}.
   *
   * @param lex  a list whose items are of type {@code String} and
   *             are the words
   * @return The Sentence
   */
  //TODO wsg2010: This should be deprecated in favor of the method below with new labels
  public static ArrayList<Word> toUntaggedList(List<String> lex) {
    ArrayList<Word> sent = new ArrayList<>();

View on GitHub (pinned to 1b7edd19c4)