stanfordnlp/CoreNLP · error · RuntimeException

Cannot find word in the text of sentence

Error message

Cannot find word  in the text of sentence 

What it means

UniversalDependenciesConverter.addSpaceAfter aligns each token of a dependency graph against the raw sentence text using text.indexOf(word, pos). If a token's word string cannot be found at/after the current position in the text, it throws RuntimeException("Cannot find word <word> in the text of sentence <graphIdx>..."). It indicates the tokenization and the raw text are out of sync.

Solutions

1. Fix the CoNLL-U file so the # text line matches the FORM tokens exactly
2. Check for Unicode normalization differences between text and word forms and normalize both
3. Handle multi-word tokens by using the surface form rather than the split word for offset search
4. Catch RuntimeException per sentence and skip/report the offending sentence instead of aborting conversion

Example fix

// before
String word = tokens.get(i).word(); // 'del' split form
int nextPos = text.indexOf(word, pos); // fails: surface text has 'del'
// after
String word = tokens.get(i).word();
if (text.indexOf(word, pos) < 0) {
  word = java.text.Normalizer.normalize(word, java.text.Normalizer.Form.NFC);
  // or fall back to the multi-word-token surface form from the CoNLL-U line
}
int nextPos = text.indexOf(word, pos);

// verify each FORM token appears in the # text line before converting
String text = sentenceText.toLowerCase(Locale.ROOT);
for (String form : forms) {
  if (!text.contains(form.toLowerCase(Locale.ROOT))) throw new CoNLLUFormatException("FORM not in text: " + form);
}

int idx = text.indexOf(word, pos);
if (idx < 0) { normalizeBothSides(); idx = text.indexOf(word, pos); }
if (idx < 0) { skipSentence(graphIdx); return; }

try { converter.convert(graph, text); } catch (RuntimeException e) { if (e.getMessage() != null && e.getMessage().startsWith("Cannot find word")) { reportMisalignedSentence(graphIdx); } else throw e; }

When it happens

Trigger: Converting CoNLL-U data where a token's word form does not appear in the sentence text at the expected offset — e.g. escaped characters, Unicode normalization differences, MWEs spanning spaces, or comments/text lines that don't match the token column.

Common situations: CoNLL-U files where # text metadata disagrees with the FORM column; multi-word tokens expanded into syntactic words (e.g. Spanish 'del' -> de+el); text with HTML entities or escaped characters; files edited so the text line no longer matches tokens.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Cannot enable POSSequence features without POS sequence…
• CoNLL-U file and tree file are not aligned.
• CoNLLUOutputter: unknown dependencies type " +…
• Could not ensure a server:
• Could not find a valid edge to remove

AI-assisted analysis of stanfordnlp/CoreNLP@1b7edd19c4 (2026-09-10). Data as JSON: /api/errors/fa3eb42afde31b7c. Report an issue: GitHub.

Appendix: source

Thrown at src/edu/stanford/nlp/trees/ud/UniversalDependenciesConverter.java:236

      } catch (Exception ex) {
        log.warn("Error running " + NER_COMBINER_NAME + " on Tree!  Not applying NER tags!");
      }
    }
  }

  /**
   * Break up a provided text input to match a tree's words,
   * using the whitespace between the words to mark the AfterAnnotation.
   * We assume a blank space at the end of the sentence
   */
  private static void addSpaceAfter(SemanticGraph sg, String text, int graphIdx) {
    List<IndexedWord> tokens = sg.vertexListSorted();
    int pos = tokens.get(0).word().length();
    for (int i = 1; i < tokens.size(); ++i) {
      String word = tokens.get(i).word();
      int nextPos = text.indexOf(word, pos);
      if (nextPos < 0) {
        throw new RuntimeException("Cannot find word " + word + " in the text of sentence " + graphIdx + "\n" + text);
      }
      tokens.get(i-1).setAfter(text.substring(pos, nextPos));
      pos = nextPos + word.length();
    }
    tokens.get(tokens.size() - 1).setAfter(" ");
  }

  /**
   * Converts a constituency tree to the English basic, enhanced, or
   * enhanced++ Universal dependencies representation, or an English basic
   * Universal dependencies tree to the enhanced or enhanced++ representation.
   * <p>
   * Command-line options:<br>
   * {@code -treeFile}: File with PTB-formatted constituency trees<br>
   * {@code -conlluFile}: File with basic dependency trees in CoNLL-U format<br>
   * {@code -textFile}: A file with text to be used as a guide for SpaceAfter (optional)<br>
   * {@code -outputRepresentation}: "basic" (default), "enhanced", or "enhanced++"<br>
   * {@code -combineMWTs}: "False" (default), "True" marks things like it's as MWT

View on GitHub (pinned to 1b7edd19c4)