{"record":{"id":"25719ec9f69a2f6e","repo":"stanfordnlp/CoreNLP","slug":"unexpected-input-many-fields","errorCode":null,"errorMessage":"Unexpected input (many fields): ","messagePattern":"Unexpected input \\(many fields\\): ","errorType":"validation","errorClass":"RuntimeIOException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/sequences/CoNLLDocumentReaderAndWriter.java","lineNumber":170,"sourceCode":"    case 4:\n      wi.setWord(bits[0]);\n      wi.setTag(bits[1]);\n      wi.set(CoreAnnotations.ChunkAnnotation.class, bits[2]);\n      wi.set(CoreAnnotations.AnswerAnnotation.class, bits[3]);\n      break;\n    case 5:\n      if (flags.useLemmaAsWord) {\n        wi.setWord(bits[1]);\n      } else {\n        wi.setWord(bits[0]);\n      }\n      wi.set(CoreAnnotations.LemmaAnnotation.class, bits[1]);\n      wi.setTag(bits[2]);\n      wi.set(CoreAnnotations.ChunkAnnotation.class, bits[3]);\n      wi.set(CoreAnnotations.AnswerAnnotation.class, bits[4]);\n      break;\n    default:\n      throw new RuntimeIOException(\"Unexpected input (many fields): \" + line);\n    }\n\n    //Value annotation is used in a lot of place in corenlp so setting here as the word itself\n    wi.set(CoreAnnotations.ValueAnnotation.class, wi.word());\n\n    // The copy to GoldAnswerAnnotation is done before the recoding is done, and so it preserves the original coding.\n    // This is important if the original coding is true, but the recoding is defective (like IOB2 to IO), since\n    // it will allow correct evaluation later.\n    wi.set(CoreAnnotations.GoldAnswerAnnotation.class, wi.get(CoreAnnotations.AnswerAnnotation.class));\n    return wi;\n  }\n\n  /** Return the coding scheme to IOB1 coding, regardless of what was used\n   *  internally (unless retainEntitySubclassification is set).\n   *  This is useful for scoring against CoNLL test output.\n   *\n   *  @param tokens List of tokens in some NER encoding\n   */","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/sequences/CoNLLDocumentReaderAndWriter.java#L152-L188","documentation":"CoNLLDocumentReaderAndWriter.makeCoreLabel parses each CoNLL line into a fixed number of whitespace-separated fields via a switch on the expected field count. A line with more fields than any configured format handles falls to default and throws RuntimeIOException('Unexpected input (many fields): ' + line).","triggerScenarios":"Reading a CoNLL-formatted file whose rows contain more tab/space-separated columns than the reader is configured for (e.g. a 6+ column CoNLL file when reader expects 5), at CoNLLDocumentReaderAndWriter.java:170 in processDocument.","commonSituations":"Feeding CoNLL-2012 or richer annotation files into a reader configured for basic CoNLL-2003 format; files with extra trailing columns; using the wrong flags (e.g. missing -readerOptions for format).","solutions":["Trim or preprocess the file so each line has exactly the number of fields the configured format expects.","Pass the appropriate reader options/flags (e.g. InputRewriteRules / format flags) so makeCoreLabel handles the file's field count.","Verify columns are not accidentally split on internal spaces (quoted fields) inflating the field count.","Check you are using the correct DocumentReaderAndWriter for the dataset (CoNLL-2003 vs CoNLL-2012)."],"exampleFix":"// before (6 fields given, 5 expected)\nNike   nike   NNP   B-NP   ORG   extra\n// after\nNike   nike   NNP   B-NP   ORG","handlingStrategy":"validation","validationCode":"// check field count of first data line against expected format\nString[] first = Files.lines(Path.of(conllFile)).filter(l -> !l.isEmpty()).findFirst().orElse(\"\").split(\"\\\\s+\");\nif (first.length > 5) throw new IllegalArgumentException(\"file has \" + first.length + \" fields; reader expects 5\");","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Match the reader (CoNLL-2003 vs CoNLL-2012) to the dataset format.","Sanitize/normalize files before loading (extra columns, embedded spaces).","Validate the first non-empty line before processing the whole file."],"tags":["java","stanford-corenlp","conll","input-format"],"backgroundTag":"invalid-argument-format","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}