stanfordnlp/CoreNLP · error · RuntimeException

Found line with label " + line + " but no tokens to…

Error message

Found line with label " + line + " but no tokens to associate with that line

What it means

BuildBinarizedDataset.extractLabels parses each input line as a sentiment label followed by the phrase tokens. A line containing only one whitespace-delimited piece means there is a label but no tokens, so it throws RuntimeException indicating the input file is malformed at that line.

Solutions

1. Open the input file and fix or delete the offending label-only line (file/line is named in the message)
2. Preprocess the file to skip lines with fewer than 2 tokens
3. Regenerate or re-download the dataset from a trusted source

Example fix

// before (bad line in input file)
1
// after
1 the actors ' expenses

List<String> bad = Files.readAllLines(input).stream()
  .filter(l -> !l.trim().isEmpty())
  .filter(l -> l.trim().split("\\s+").length < 2)
  .collect(toList());
if (!bad.isEmpty()) throw new IllegalArgumentException("Label-only lines: " + bad);

static boolean hasTokens(String line) { return line.trim().split("\\s+").length >= 2; }

try {
  BuildBinarizedDataset.main(args);
} catch (RuntimeException e) {
  log.error("Malformed dataset line: " + e.getMessage());
}

When it happens

Trigger: A line in the input dataset (e.g. Stanford Sentiment Treebank-style label+phrase file) contains a bare label like "1" with no following phrase tokens.

Common situations: Blank-ish lines containing only whitespace plus a number; truncated files; copy/paste errors when preparing the training data; files with a trailing label-only line.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• Argument array lengths differ
• Array lengths don't match
• attempt to get word when sentence and lattice are null!
• Attempted to parse empty/null tag
• Bad data format:

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/f04e5bee128cbb35. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/sentiment/BuildBinarizedDataset.java:66

    if (tree.isLeaf()) {
      return;
    }

    for (Tree child : tree.children()) {
      setPredictedLabels(child);
    }

    tree.label().setValue(Integer.toString(RNNCoreAnnotations.getPredictedClass(tree)));
  }

  public static void extractLabels(Map<Pair<Integer, Integer>, String> spanToLabels, List<HasWord> tokens, String line) {
    String[] pieces = line.trim().split("\\s+");
    if (pieces.length == 0) {
      return;
    }
    if (pieces.length == 1) {
      String error = "Found line with label " + line + " but no tokens to associate with that line";
      throw new RuntimeException(error);
    }

    //TODO: BUG: The pieces are tokenized differently than the splitting, e.g., on possessive markers as in "actors' expenses"
    for (int i = 0; i < tokens.size() - pieces.length + 2; ++i) {
      boolean found = true;
      for (int j = 1; j < pieces.length; ++j) {
        if (!tokens.get(i + j - 1).word().equals(pieces[j])) {
          found = false;
          break;
        }
      }
      if (found) {
        spanToLabels.put(new Pair<>(i, i + pieces.length - 1), pieces[0]);
      }
    }
  }

  public static boolean setSpanLabel(Tree tree, Pair<Integer, Integer> span, String value) {

View on GitHub (pinned to 1b7edd19c4)