{"record":{"id":"c67505ad82fb2d8b","repo":"stanfordnlp/CoreNLP","slug":"sentence-tosentence-lengths-differ","errorCode":null,"errorMessage":"Sentence.toSentence: lengths differ","messagePattern":"Sentence\\.toSentence: lengths differ","errorType":"exception","errorClass":"java.lang.IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ling/SentenceUtils.java","lineNumber":39,"sourceCode":"  private SentenceUtils() {} // static methods\n\n  /**\n   * Create an ArrayList as a list of {@code TaggedWord} from two\n   * lists of {@code String}, one for the words, and the second for\n   * the tags.\n   *\n   * @param lex  a list whose items are of type {@code String} and\n   *             are the words\n   * @param tags a list whose items are of type {@code String} and\n   *             are the tags\n   * @return The Sentence\n   */\n  public static ArrayList<TaggedWord> toTaggedList(List<String> lex, List<String> tags) {\n    ArrayList<TaggedWord> sent = new ArrayList<>();\n    int ls = lex.size();\n    int ts = tags.size();\n    if (ls != ts) {\n      throw new IllegalArgumentException(\"Sentence.toSentence: lengths differ\");\n    }\n    for (int i = 0; i < ls; i++) {\n      sent.add(new TaggedWord(lex.get(i), tags.get(i)));\n    }\n    return sent;\n  }\n\n  /**\n   * Create an ArrayList as a list of {@code Word} from a\n   * list of {@code String}.\n   *\n   * @param lex  a list whose items are of type {@code String} and\n   *             are the words\n   * @return The Sentence\n   */\n  //TODO wsg2010: This should be deprecated in favor of the method below with new labels\n  public static ArrayList<Word> toUntaggedList(List<String> lex) {\n    ArrayList<Word> sent = new ArrayList<>();","sourceCodeStart":21,"sourceCodeEnd":57,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ling/SentenceUtils.java#L21-L57","documentation":"SentenceUtils.toTaggedList builds a TaggedWord list by pairing the lex (words) and tags lists element-by-element. If the two input lists have different sizes, the method throws IllegalArgumentException(\"Sentence.toSentence: lengths differ\") because every word must have exactly one tag. This is an input-validation guard, not a library bug.","triggerScenarios":"Calling SentenceUtils.toTaggedList(lex, tags) (or the equivalent Sentence.toSentence overloads) with lex.size() != tags.size(), e.g. after a tokenization step dropped or added items on one side.","commonSituations":"Piping tokens from a tokenizer into a POS tagger where empty tokens were filtered from the word list but not the tag list; reading parallel word/tag data from a file with a ragged last line; off-by-one trimming of whitespace-only tokens.","solutions":["Validate lex.size() == tags.size() before the call and fix whichever list is wrong.","Ensure both lists come from the same tokenization pass — don't filter or split one list independently of the other.","If lists may legitimately differ, zip to the min length yourself and construct TaggedWord pairs in a loop instead.","Log both sizes at the call site to identify which pipeline stage desynchronized the lists."],"exampleFix":"// before\nList<TaggedWord> sent = SentenceUtils.toTaggedList(lex, tags); // throws if sizes differ\n// after\nif (lex.size() != tags.size()) {\n  throw new IllegalArgumentException(\"lex=\" + lex.size() + \" tags=\" + tags.size());\n}\nList<TaggedWord> sent = SentenceUtils.toTaggedList(lex, tags);","handlingStrategy":"validation","validationCode":"if (lex == null || tags == null || lex.size() != tags.size()) {\n  throw new IllegalArgumentException(\n      \"toTaggedList requires equal sizes: lex=\" + (lex == null ? -1 : lex.size())\n      + \" tags=\" + (tags == null ? -1 : tags.size()));\n}\nList<TaggedWord> sent = SentenceUtils.toTaggedList(lex, tags);","typeGuard":"boolean isParallel(List<?> a, List<?> b) {\n  return a != null && b != null && a.size() == b.size();\n}","tryCatchPattern":"try {\n  return SentenceUtils.toTaggedList(lex, tags);\n} catch (IllegalArgumentException e) {\n  if (!e.getMessage().contains(\"lengths differ\")) throw e;\n  log.error(\"lex.size=\" + lex.size() + \" tags.size=\" + tags.size());\n  int n = Math.min(lex.size(), tags.size());\n  return SentenceUtils.toTaggedList(lex.subList(0, n), tags.subList(0, n));\n}","preventionTips":["Always derive the word and tag lists from the same tokenization pass.","Assert equal list sizes at pipeline boundaries with a descriptive error including both sizes.","Don't filter/trim one list independently of its parallel list.","Keep word/tag pairs together in a single structure (e.g. List<String[]>) until the moment you call toTaggedList."],"tags":["java","nlp","illegal-argument","sentence","validation"],"backgroundTag":"schema-validation-failed","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}