stanfordnlp/CoreNLP · error · IllegalArgumentException

File line # too short

Error message

File ${filename} line #${linesRead} too short

What it means

TSVTaggedFileReader splits each non-empty line of a TSV training/test file on tabs and needs entries at both wordColumn and tagColumn indices. If a line has fewer tab-separated fields than required, primeNext() throws IllegalArgumentException identifying the file and line number.

Solutions

1. Open the file at the reported line number and fix or remove the short line.
2. Confirm wordColumn/tagColumn options match the actual number of columns in your file.
3. Regenerate or re-export the file ensuring every row has all required tab-separated fields (use a TSV-aware tool, not a spreadsheet that may alter tabs).

Example fix

// before
// config: -wordColumn 1 -tagColumn 2, but file rows have only 2 columns
the	DT
// after (add missing column or fix config)
1	the	DT

// Java: pre-check file before tagging
List<String> bad = new ArrayList<>();
int ln = 0;
for (String line : Files.readAllLines(path)) {
  ln++;
  if (!line.trim().isEmpty() && line.split("\t").length <= Math.max(wordColumn, tagColumn))
    bad.add("line " + ln);
}

When it happens

Trigger: Reading a TSV file where a line (or the last line before EOF, or a sentence-final line) contains fewer tab-separated columns than max(wordColumn, tagColumn)+1, given the configured columns.

Common situations: Files exported with spaces instead of tabs, rows with missing trailing tag columns, editor tools converting tabs, blank-ish lines with stray content, or mismatched column configuration (e.g., columns set to 1,2 but the file has 2 fields).

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Data format error: can't find delimiter
• CTBunk file not found
• CTBunk I/O error
• Error on line
• Found a non-empty line in a tsurgeon section after reading…

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/dc01a2103adb664c. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/tagger/io/TSVTaggedFileReader.java:88

        line = reader.readLine();
        ++linesRead;
      } catch (IOException e) {
        throw new RuntimeException(e);
      }
      if (line == null) {
        next = null;
        return;
      }
    }
    // we hit something with text, so now we read one line at a time
    // until we hit the next blank line.  the next blank line (or EOF)
    // ends the sentence.
    next = new ArrayList<>();
    while (line != null && ! line.trim().isEmpty()) {
      if (!(usesComments && line.startsWith("#"))) {
        String[] pieces = line.split("\t");
        if (pieces.length <= wordColumn || pieces.length <= tagColumn) {
          throw new IllegalArgumentException("File " + filename + " line #" +
                                             linesRead + " too short");
        }
        if (!(skipMWT && pieces[0].matches("[0-9]+-[0-9]+"))) {
          String word = pieces[wordColumn];
          String tag = pieces[tagColumn];
          next.add(new TaggedWord(word, tag));
        }
      }
      try {
        line = reader.readLine();
        ++linesRead;
      } catch (IOException e) {
        throw new RuntimeException(e);
      }
    }
  }

  @Override

View on GitHub (pinned to 1b7edd19c4)