{"record":{"id":"1a088122589bd176","repo":"stanfordnlp/CoreNLP","slug":"invalid-line-line","errorCode":null,"errorMessage":"INVALID LINE: \"${line}\"","messagePattern":"INVALID LINE: \"(.+?)\"","errorType":"validation","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/ie/machinereading/BasicEntityExtractor.java","lineNumber":480,"sourceCode":"    \t\tos = new PrintStream(new FileOutputStream(dir + File.separator + docid + \".conll\"));\n    \t}\n      List<CoreLabel> labeledSentence = AnnotationUtils.sentenceEntityMentionsToCoreLabels(sentence, true, null, null, useSubTypes, alreadyBIO);\n      assert(labeledSentence != null);\n\n      String prev = null;\n      for(CoreLabel word: labeledSentence) {\n        String w = word.word().replaceAll(\"[ \\t\\n]+\", \"_\");\n        String t = word.get(CoreAnnotations.PartOfSpeechAnnotation.class);\n        String l = word.get(CoreAnnotations.AnswerAnnotation.class);\n        String nl = l;\n        if(! alreadyBIO && ! l.equals(\"O\")){\n          if(prev != null && l.equals(prev)) nl = \"I-\" + l;\n          else nl = \"B-\" + l;\n        }\n        String line = w + \" \" + t + \" \" + nl;\n        String [] toks = line.split(\"[ \\t\\n]+\");\n        if(toks.length != 3){\n          throw new RuntimeException(\"INVALID LINE: \\\"\" + line + \"\\\"\");\n        }\n        os.printf(\"%s %s %s\\n\", w, t, nl);\n        prev = l;\n      }\n      os.println();\n    }\n    if(os != null){\n    \tos.close();\n    }\n  }\n\n  public static void saveCoNLL(PrintStream os, List<List<CoreLabel>> sentences, boolean alreadyBIO) {\n    os.println(\"-DOCSTART- -X- O\\n\");\n    for(List<CoreLabel> sent: sentences){\n      String prev = null;\n      for(CoreLabel word: sent) {\n        String w = word.word().replaceAll(\"[ \\t\\n]+\", \"_\");\n        String t = word.get(CoreAnnotations.PartOfSpeechAnnotation.class);","sourceCodeStart":462,"sourceCodeEnd":498,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/ie/machinereading/BasicEntityExtractor.java#L462-L498","documentation":"In saveCoNLLFiles, each token is written as a CoNLL line \"word tag label\" and is re-split to verify it decomposes into exactly 3 whitespace-separated tokens. If the word or tag itself contains whitespace, the reconstructed line splits into more than 3 fields and the code throws rather than writing malformed CoNLL output.","triggerScenarios":"Saving annotations where a word (w) or tag (t) string contains an internal space, tab, or newline: line.split(\"[ \\t\\n]+\") then yields toks.length != 3.","commonSituations":"Tokenization mismatches — a 'word' that still contains whitespace (untokenized input, annotation spans joined with spaces); tags with stray whitespace from external NER outputs; corrupted annotations read from a dirty source file.","solutions":["Sanitize w and t before formatting: replace whitespace characters with a placeholder (e.g. '_') or re-split them.","Assert upstream that tokens are single whitespace-free strings before calling saveCoNLLFiles.","Skip or log-and-continue on offending tokens instead of formatting them into the CoNLL output."],"exampleFix":"// before\nString line = w + \" \" + t + \" \" + nl;\n\n// after\nString wSafe = w.replaceAll(\"[ \\\\t\\\\n]+\", \"_\");\nString tSafe = t.replaceAll(\"[ \\\\t\\\\n]+\", \"_\");\nString line = wSafe + \" \" + tSafe + \" \" + nl;","handlingStrategy":"validation","validationCode":"// Java: sanitize before saving\nassert !w.matches(\".*[ \\\\t\\\\n].*\") : \"word contains whitespace: \" + w;\nassert !t.matches(\".*[ \\\\t\\\\n].*\") : \"tag contains whitespace: \" + t;\nextractor.saveCoNLLFiles(...);","typeGuard":null,"tryCatchPattern":"// Java\ntry {\n  extractor.saveCoNLLFiles(testFile, docs, biased); \n} catch (RuntimeException e) {\n  if (e.getMessage().startsWith(\"INVALID LINE\")) {\n    log.severe(\"CoNLL export hit a token with embedded whitespace: \" + e.getMessage());\n    // re-sanitize corpus and retry\n  } else throw e;\n}","preventionTips":["Run tokens through a tokenizer so words never contain whitespace.","Trim/normalize tags coming from external annotation tools.","Validate a sample of the corpus before full export."],"tags":["java","nlp","conll","serialization"],"backgroundTag":"invalid-argument-format","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}