stanfordnlp/CoreNLP · error · IOException

Error on line

Error message

Error on line 

What it means

BaseLexicon.readGrammaticalStructure... actually this is the lexicon text-loading path: while reading a lexicon file line by line, any RuntimeException from parsing a line (malformed fields, bad number in the weight field, missing columns) is wrapped into an IOException whose message includes the line number and the offending line, to make the corrupt input locatable.

Solutions

1. Inspect the reported line number in the lexicon file and fix the malformed line (5 tab/whitespace-separated fields: tag, seen-flag, word, ..., count)
2. Check the field order matches the writer's format (tag, seen/UNSEEN, word, ..., weight)
3. Remove header/comment/empty lines from the lexicon file
4. Verify you are loading the right file type (text lexicon vs serialized model)

Example fix

// before (bad line 42: only 4 fields)
NN  SEEN  dog  100
// after
NN  SEEN  dog  UNK  100

// validate lexicon lines before loading
int lineNum = 0;
for (String line : Files.readAllLines(lexiconPath)) {
  lineNum++;
  String[] fields = line.trim().split("\\s+");
  if (fields.length != 5) throw new IOException("lexicon line " + lineNum + " has " + fields.length + " fields, expected 5");
  Double.parseDouble(fields[4]);
}

try {
  lexicon.loadFile(lexiconPath);
} catch (IOException e) {
  if (e.getMessage().startsWith("Error on line")) {
    System.err.println("Fix the cited lexicon line: " + e.getMessage());
  }
  throw e;
}

When it happens

Trigger: Loading a lexicon text file (e.g. via -textFile lexicon or readData) where a line does not have the expected 5 fields SEEN/UNSEEN word tag count — e.g. Double.parseDouble(fields[4]) fails or fields[2]/fields[0] don't split into word/tag correctly.

Common situations: Hand-edited lexicon files, files with header lines or blank/truncated lines, wrong file passed as the lexicon (e.g. a model binary given as text lexicon), wrong column order.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Error loading classifier from
• edu.stanford.nlp.io.RuntimeIOException
• Error creating data exporter
• Error reading saved links
• Error creating data exporter

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/f83bebd171df28de. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/parser/lexparser/BaseLexicon.java:789

    // all lines have one tagging with raw count per line
    line = in.readLine();
    Pattern p = Pattern.compile("^smooth\\[([0-9])\\] = (.*)$");
    while (line != null && line.length() > 0) {
      try {
        Matcher m = p.matcher(line);
        if (m.matches()) {
          int i = Integer.parseInt(m.group(1));
          smooth[i] = Double.parseDouble(m.group(2));
        } else {
          // split on spaces, quote with doublequote, and escape with backslash
          String[] fields = StringUtils.splitOnCharWithQuoting(line, ' ', '\"', '\\');
          // System.out.println("fields:\n" + fields[0] + "\n" + fields[1] +
          // "\n" + fields[2] + "\n" + fields[3] + "\n" + fields[4]);
          boolean seen = fields[3].equals(SEEN);
          addTagging(seen, new IntTaggedWord(fields[2], fields[0], wordIndex, tagIndex), Double.parseDouble(fields[4]));
        }
      } catch (RuntimeException e) {
        throw new IOException("Error on line " + lineNum + ": " + line, e);
      }
      lineNum++;
      line = in.readLine();
    }
    initRulesWithWord();
  }

  /**
   * Writes out data from this Object to the Writer w. Rules are separated by
   * newline, and rule elements are delimited by \t.
   */
  @Override
  public void writeData(Writer w) throws IOException {
    PrintWriter out = new PrintWriter(w);

    for (Map.Entry<IntTaggedWord, Double> entry : seenCounter.entrySet()) {
      out.println(entry.getKey().toLexicalEntry(wordIndex, tagIndex) + " SEEN " + entry.getValue());
    }

View on GitHub (pinned to 1b7edd19c4)