stanfordnlp/CoreNLP · error · RuntimeException

INVALID LINE

Error message

INVALID LINE: "${line}"

What it means

In saveCoNLLFiles, each token is written as a CoNLL line "word tag label" and is re-split to verify it decomposes into exactly 3 whitespace-separated tokens. If the word or tag itself contains whitespace, the reconstructed line splits into more than 3 fields and the code throws rather than writing malformed CoNLL output.

Solutions

1. Sanitize w and t before formatting: replace whitespace characters with a placeholder (e.g. '_') or re-split them.
2. Assert upstream that tokens are single whitespace-free strings before calling saveCoNLLFiles.
3. Skip or log-and-continue on offending tokens instead of formatting them into the CoNLL output.

Example fix

// before
String line = w + " " + t + " " + nl;

// after
String wSafe = w.replaceAll("[ \\t\\n]+", "_");
String tSafe = t.replaceAll("[ \\t\\n]+", "_");
String line = wSafe + " " + tSafe + " " + nl;

// Java: sanitize before saving
assert !w.matches(".*[ \\t\\n].*") : "word contains whitespace: " + w;
assert !t.matches(".*[ \\t\\n].*") : "tag contains whitespace: " + t;
extractor.saveCoNLLFiles(...);

// Java
try {
  extractor.saveCoNLLFiles(testFile, docs, biased); 
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("INVALID LINE")) {
    log.severe("CoNLL export hit a token with embedded whitespace: " + e.getMessage());
    // re-sanitize corpus and retry
  } else throw e;
}

When it happens

Trigger: Saving annotations where a word (w) or tag (t) string contains an internal space, tab, or newline: line.split("[ \t\n]+") then yields toks.length != 3.

Common situations: Tokenization mismatches — a 'word' that still contains whitespace (untokenized input, annotation spans joined with spaces); tags with stray whitespace from external NER outputs; corrupted annotations read from a dirty source file.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• is not a legal LogPrior.
• Cannot find matching labelled span for
• Error extracting labelled spans for column :
• Unexpected number of field , expected >= for line (,):
• RuntimeIOException wrapping IOException

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/1a088122589bd176. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/ie/machinereading/BasicEntityExtractor.java:480

    		os = new PrintStream(new FileOutputStream(dir + File.separator + docid + ".conll"));
    	}
      List<CoreLabel> labeledSentence = AnnotationUtils.sentenceEntityMentionsToCoreLabels(sentence, true, null, null, useSubTypes, alreadyBIO);
      assert(labeledSentence != null);

      String prev = null;
      for(CoreLabel word: labeledSentence) {
        String w = word.word().replaceAll("[ \t\n]+", "_");
        String t = word.get(CoreAnnotations.PartOfSpeechAnnotation.class);
        String l = word.get(CoreAnnotations.AnswerAnnotation.class);
        String nl = l;
        if(! alreadyBIO && ! l.equals("O")){
          if(prev != null && l.equals(prev)) nl = "I-" + l;
          else nl = "B-" + l;
        }
        String line = w + " " + t + " " + nl;
        String [] toks = line.split("[ \t\n]+");
        if(toks.length != 3){
          throw new RuntimeException("INVALID LINE: \"" + line + "\"");
        }
        os.printf("%s %s %s\n", w, t, nl);
        prev = l;
      }
      os.println();
    }
    if(os != null){
    	os.close();
    }
  }

  public static void saveCoNLL(PrintStream os, List<List<CoreLabel>> sentences, boolean alreadyBIO) {
    os.println("-DOCSTART- -X- O\n");
    for(List<CoreLabel> sent: sentences){
      String prev = null;
      for(CoreLabel word: sent) {
        String w = word.word().replaceAll("[ \t\n]+", "_");
        String t = word.get(CoreAnnotations.PartOfSpeechAnnotation.class);

View on GitHub (pinned to 1b7edd19c4)