stanfordnlp/CoreNLP · error · Exception

Line doesn't contain two tokens

Error message

Line doesn't contain two tokens: ${line}

What it means

readInput() reads SimpleGoodTuring's stdin input format, which expects each line to contain exactly two whitespace-separated tokens: the count r and the number of types n. Any line with a different token count throws a generic Exception naming the offending line.

Solutions

1. Fix the input so every line has exactly two whitespace-separated integer tokens (r and n).
2. Strip header/comment/blank lines from the input before piping it in.
3. Pre-validate each line in your own code (split on whitespace, check length == 2) before feeding it to the program.

Example fix

// before
echo "5 3 extra" | java SimpleGoodTuring ...
// after
echo "5 3" | java SimpleGoodTuring ...

// Java: validate lines before piping
boolean valid = line.trim().split("\\s+").length == 2;

When it happens

Trigger: Feeding the main()/input() entry point stdin lines that don't have exactly two whitespace-separated integers, e.g. blank-but-not-empty lines, extra columns, comments, or a header row.

Common situations: Piping a file with a header or trailing annotation lines into the program, copy-pasting data with stray whitespace or an index column, or Windows line endings combining tokens unexpectedly.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• attempt to get word when sentence and lattice are null!
• Bad number put into wordToNumber. Word is: \"" + input +…
• Bad number put into wordToNumber. Word is: \"" + curPart +…
• Can't return head of null or leaf Tree.
• CANNOT EVEN CREATE ARRAYS OF ORIGINAL SIZE!!

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/8703966274829c24. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/stats/SimpleGoodTuring.java:203

  }


  // static methods -------------------------------------------------------------

  /**
   * Reads from STDIN a sequence of lines, each containing two integers,
   * separated by whitespace.  Returns a pair of int arrays containing the
   * values read.
   */
  private static int[][] readInput() throws Exception {
    List<Integer> rVals = new ArrayList<>();
    List<Integer> nVals = new ArrayList<>();
    BufferedReader in = new BufferedReader(new InputStreamReader(System.in));
    String line;
    while ((line = in.readLine()) != null) {
      String[] tokens = line.trim().split("\\s+");
      if (tokens.length != 2)
        throw new Exception("Line doesn't contain two tokens: " + line);
      Integer r = Integer.valueOf(tokens[0]);
      Integer n = Integer.valueOf(tokens[1]);
      rVals.add(r);
      nVals.add(n);
    }
    in.close();
    int[][] result = new int[2][];
    result[0] = integerList2IntArray(rVals);
    result[1] = integerList2IntArray(nVals);
    return result;
  }

  /**
   * Helper to readInput().
   */
  private static int[] integerList2IntArray(List<Integer> integers) {
    int[] ints = new int[integers.size()];
    int i = 0;

View on GitHub (pinned to 1b7edd19c4)