stanfordnlp/CoreNLP · error · RuntimeException

no token in tree: tokens

Error message

no token %d in tree:
%s
tokens:
%s

What it means

TimexTreeAnnotator's endOffset reads the tree label's EndIndexAnnotation and uses it to index into the token list; if endToken exceeds the token list size, the referenced token does not exist and it throws RuntimeException 'no token %d in tree'. This indicates the tree's token indices are out of sync with the sentence tokens.

Solutions

1. Ensure the tree and the token list come from the same sentence/annotation with consistent indexing
2. Recompute the parse (or at least the index annotations) after any re-tokenization
3. Guard by checking tree label EndIndexAnnotation against tokens.size() before calling endOffset
4. Catch RuntimeException, log tree+tokens, and skip the misaligned Timex

Example fix

// before
int end = endOffset(tree, tokens);
// after
Integer endTok = ((CoreMap) tree.label()).get(CoreAnnotations.EndIndexAnnotation.class);
if (endTok == null || endTok > tokens.size()) {
  throw new IllegalArgumentException("tree/token misalignment");
}
int end = endOffset(tree, tokens);

Integer endTok = ((CoreMap) tree.label()).get(CoreAnnotations.EndIndexAnnotation.class); if (endTok != null && endTok <= tokens.size()) { /* safe */ }

boolean treeMatchesTokens(Tree tree, List<CoreLabel> tokens) { Integer e = ((CoreMap) tree.label()).get(CoreAnnotations.EndIndexAnnotation.class); return e != null && e > 0 && e <= tokens.size(); }

try { int end = endOffset(tree, tokens); } catch (RuntimeException e) { log.warn("tree/token misalignment, skipping"); }

When it happens

Trigger: Annotating a tree whose CoreMap label carries an EndIndexAnnotation larger than the number of tokens — typically when the parse tree and the token list come from different sentences/documents, or offsets were built over a different segmentation.

Common situations: Pipelines where trees are re-used across sentences, or where tokenization changed after tree indices were computed (e.g. splitting, filter, or re-tokenizing the sentence).

Related errors

• Tokenizer unable to find text in annotation
• -headIndex of is out of bounds for a phrase with words
• unexpected match type
• Must supply a target label to compute precision and recall…
• Cannot compute precision and recall on unlabelled dataset…

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/64c312c0375ff1c0. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/time/TimexTreeAnnotator.java:92

        if (subtree != null) {
          timexAnn.set(TreeCoreAnnotations.TreeAnnotation.class, subtree);
        }
      }
    }
  }
  
  private static int beginOffset(Tree tree, List<CoreLabel> tokens) {
    CoreMap label = (CoreMap)tree.label();
    int beginToken = label.get(CoreAnnotations.BeginIndexAnnotation.class);
    return beginOffset(tokens.get(beginToken));
  }
  
  private static int endOffset(Tree tree, List<CoreLabel> tokens) {
    CoreMap label = (CoreMap)tree.label();
    int endToken = label.get(CoreAnnotations.EndIndexAnnotation.class);
    if (endToken > tokens.size()) {
      String msg = "no token %d in tree:\n%s\ntokens:\n%s";
      throw new RuntimeException(String.format(msg, endToken - 1, tree, tokens));
    }
    return endOffset(tokens.get(endToken - 1));
  }
  
  private static int beginOffset(CoreMap map) {
    return map.get(CoreAnnotations.CharacterOffsetBeginAnnotation.class);
  }
  
  private static int endOffset(CoreMap map) {
    return map.get(CoreAnnotations.CharacterOffsetEndAnnotation.class);
  }

  @Override
  public Set<Class<? extends CoreAnnotation>> requires() {
    return Collections.unmodifiableSet(new ArraySet<>(Arrays.asList(
        CoreAnnotations.TextAnnotation.class,
        CoreAnnotations.TokensAnnotation.class,
        CoreAnnotations.CharacterOffsetBeginAnnotation.class,

View on GitHub (pinned to 1b7edd19c4)