{"record":{"id":"64c312c0375ff1c0","repo":"stanfordnlp/CoreNLP","slug":"no-token-d-in-tree-s-tokens-s","errorCode":null,"errorMessage":"no token %d in tree:\n%s\ntokens:\n%s","messagePattern":"no token (.+?) in tree:\n(.+?)\ntokens:\n(.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/edu/stanford/nlp/time/TimexTreeAnnotator.java","lineNumber":92,"sourceCode":"        if (subtree != null) {\n          timexAnn.set(TreeCoreAnnotations.TreeAnnotation.class, subtree);\n        }\n      }\n    }\n  }\n  \n  private static int beginOffset(Tree tree, List<CoreLabel> tokens) {\n    CoreMap label = (CoreMap)tree.label();\n    int beginToken = label.get(CoreAnnotations.BeginIndexAnnotation.class);\n    return beginOffset(tokens.get(beginToken));\n  }\n  \n  private static int endOffset(Tree tree, List<CoreLabel> tokens) {\n    CoreMap label = (CoreMap)tree.label();\n    int endToken = label.get(CoreAnnotations.EndIndexAnnotation.class);\n    if (endToken > tokens.size()) {\n      String msg = \"no token %d in tree:\\n%s\\ntokens:\\n%s\";\n      throw new RuntimeException(String.format(msg, endToken - 1, tree, tokens));\n    }\n    return endOffset(tokens.get(endToken - 1));\n  }\n  \n  private static int beginOffset(CoreMap map) {\n    return map.get(CoreAnnotations.CharacterOffsetBeginAnnotation.class);\n  }\n  \n  private static int endOffset(CoreMap map) {\n    return map.get(CoreAnnotations.CharacterOffsetEndAnnotation.class);\n  }\n\n  @Override\n  public Set<Class<? extends CoreAnnotation>> requires() {\n    return Collections.unmodifiableSet(new ArraySet<>(Arrays.asList(\n        CoreAnnotations.TextAnnotation.class,\n        CoreAnnotations.TokensAnnotation.class,\n        CoreAnnotations.CharacterOffsetBeginAnnotation.class,","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/stanfordnlp/CoreNLP/blob/1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a/src/edu/stanford/nlp/time/TimexTreeAnnotator.java#L74-L110","documentation":"TimexTreeAnnotator's endOffset reads the tree label's EndIndexAnnotation and uses it to index into the token list; if endToken exceeds the token list size, the referenced token does not exist and it throws RuntimeException 'no token %d in tree'. This indicates the tree's token indices are out of sync with the sentence tokens.","triggerScenarios":"Annotating a tree whose CoreMap label carries an EndIndexAnnotation larger than the number of tokens — typically when the parse tree and the token list come from different sentences/documents, or offsets were built over a different segmentation.","commonSituations":"Pipelines where trees are re-used across sentences, or where tokenization changed after tree indices were computed (e.g. splitting, filter, or re-tokenizing the sentence).","solutions":["Ensure the tree and the token list come from the same sentence/annotation with consistent indexing","Recompute the parse (or at least the index annotations) after any re-tokenization","Guard by checking tree label EndIndexAnnotation against tokens.size() before calling endOffset","Catch RuntimeException, log tree+tokens, and skip the misaligned Timex"],"exampleFix":"// before\nint end = endOffset(tree, tokens);\n// after\nInteger endTok = ((CoreMap) tree.label()).get(CoreAnnotations.EndIndexAnnotation.class);\nif (endTok == null || endTok > tokens.size()) {\n  throw new IllegalArgumentException(\"tree/token misalignment\");\n}\nint end = endOffset(tree, tokens);","handlingStrategy":"type-guard","validationCode":"Integer endTok = ((CoreMap) tree.label()).get(CoreAnnotations.EndIndexAnnotation.class); if (endTok != null && endTok <= tokens.size()) { /* safe */ }","typeGuard":"boolean treeMatchesTokens(Tree tree, List<CoreLabel> tokens) { Integer e = ((CoreMap) tree.label()).get(CoreAnnotations.EndIndexAnnotation.class); return e != null && e > 0 && e <= tokens.size(); }","tryCatchPattern":"try { int end = endOffset(tree, tokens); } catch (RuntimeException e) { log.warn(\"tree/token misalignment, skipping\"); }","preventionTips":["Never mix trees and token lists from different sentences or tokenizations","Re-run annotation after any re-tokenization or sentence splitting","Validate index annotations against token list size in preprocessing"],"tags":["java","index-out-of-range","annotation"],"backgroundTag":"index-out-of-bounds","analyzedSha":"1b7edd19c4d0d7b1f13a2591425b9b60a0b1af7a","analyzedAt":"2026-09-10T02:24:07.274Z","contentChangedAt":"2026-09-10T02:24:07.274Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}