{"record":{"id":"c576c8199fec3af7","repo":"stanfordnlp/CoreNLP","slug":"expected-tagged-words","errorCode":null,"errorMessage":"Expected tagged words","messagePattern":"Expected tagged words","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/parser/shiftreduce/ShiftReduceParser.java","lineNumber":245,"sourceCode":"    return initialStateFromTaggedSentence(tree.taggedYield());\n  }\n\n  public static State initialStateFromTaggedSentence(List<? extends HasWord> words) {\n    List<Tree> preterminals = Generics.newArrayList();\n    for (int index = 0; index < words.size(); ++index) {\n      HasWord hw = words.get(index);\n\n      CoreLabel wordLabel;\n      String tag;\n      if (hw instanceof CoreLabel) {\n        wordLabel = (CoreLabel) hw;\n        tag = wordLabel.tag();\n      } else {\n        wordLabel = new CoreLabel();\n        wordLabel.setValue(hw.word());\n        wordLabel.setWord(hw.word());\n        if (!(hw instanceof HasTag)) {\n          throw new IllegalArgumentException(\"Expected tagged words\");\n        }\n        tag = ((HasTag) hw).tag();\n        wordLabel.setTag(tag);\n      }\n      if (tag == null) {\n        throw new IllegalArgumentException(\"Input word not tagged\");\n      }\n      CoreLabel tagLabel = new CoreLabel();\n      tagLabel.setValue(tag);\n\n      // Index from 1.  Tools downstream from the parser expect that\n      // Internally this parser uses the index, so we have to\n      // overwrite incorrect indices if the label is already indexed\n      wordLabel.setIndex(index + 1);\n      tagLabel.setIndex(index + 1);\n\n      LabeledScoredTreeNode wordNode = new LabeledScoredTreeNode(wordLabel);\n      LabeledScoredTreeNode tagNode = new LabeledScoredTreeNode(tagLabel);","sourceCodeStart":227,"sourceCodeEnd":263,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/parser/shiftreduce/ShiftReduceParser.java#L227-L263","documentation":"ShiftReduceParser.initialStateFromTaggedSentence builds the initial parser state from input words. Each word must carry a POS tag; when a word is not a HasTag instance (and isn't already a TaggedWord/CoreLabel with a tag), the parser cannot obtain a tag and throws IllegalArgumentException(\"Expected tagged words\").","triggerScenarios":"Calling initialStateFromTaggedSentence (directly or via initialStateFromGoldTagTree / parse(List<? extends HasWord>)) with a sentence of plain Word/String tokens that do not implement HasTag.","commonSituations":"Tokenizing with a plain whitespace or Word-based tokenizer instead of a PTBTokenizer producing CoreLabels, then feeding tokens straight to the parser; forgetting to run a POS tagger on the token list.","solutions":["Run a POS tagger over the sentence so each token has a tag, and pass TaggedWord/CoreLabel objects","Construct the sentence as List<TaggedWord> with setTag on each word before calling the parser","Use the parser's documented pipeline (tokenizer + tagger) instead of hand-built Word lists"],"exampleFix":"// before\nList<Word> words = Arrays.asList(new Word(\"The\"), new Word(\"dog\"));\nState s = parser.initialStateFromTaggedSentence(words);\n\n// after\nList<TaggedWord> words = Arrays.asList(new TaggedWord(\"The\", \"DT\"), new TaggedWord(\"dog\", \"NN\"));\nState s = parser.initialStateFromTaggedSentence(words);","handlingStrategy":"validation","validationCode":"for (HasWord hw : sentence) {\n  if (!(hw instanceof HasTag) || ((HasTag) hw).tag() == null) {\n    throw new IllegalArgumentException(\"Word '\" + hw.word() + \"' is not tagged\");\n  }\n}","typeGuard":"boolean isTagged(HasWord hw) {\n  return hw instanceof HasTag && ((HasTag) hw).tag() != null;\n}","tryCatchPattern":"try {\n  state = parser.initialStateFromTaggedSentence(sentence);\n} catch (IllegalArgumentException e) {\n  state = parser.initialStateFromTaggedSentence(tagger.tagSentence(rawTokens));\n}","preventionTips":["Standardize on TaggedWord/CoreLabel token lists from the tagger","Never feed raw tokenizer Words directly to the parser","Assert tags present before parsing in test harnesses"],"tags":["parser","input-validation","pos-tags","type-mismatch"],"backgroundTag":"type-mismatch","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}