languagetool-org/languagetool · error

File with example sentences not found:

Error message

File with example sentences not found: 

What it means

ConfusionRuleEvaluator.getRelevantSentences expects each input directory to contain a per-token corpus file named <token>.txt. When the directory exists but the file for the current confusion token is missing, it throws this RuntimeException. It is a dev-tooling corpus-preparation check: without the example sentence file, no evaluation data can be gathered for that token.

Source

Thrown at languagetool-dev/src/main/java/org/languagetool/dev/bigdata/ConfusionRuleEvaluator.java:214

        //System.out.printf(ENGLISH, "Precision:    %.3f (%d false positives)\n", precision, evalValues.falsePositives);
        //System.out.printf(ENGLISH, "Recall:       %.3f (%d false negatives)\n", recall, evalValues.falseNegatives);
        //double fMeasure = FMeasure.getWeightedFMeasure(precision, recall);
        //System.out.printf(ENGLISH, "F-measure:    %.3f (beta=0.5)\n", fMeasure);
        //System.out.printf(ENGLISH, "Good Matches: %d (true positives)\n", evalValues.truePositives);
        //System.out.printf(ENGLISH, "All matches:  %d\n", evalValues.truePositives + evalValues.falsePositives);
        System.out.printf(summary + "\n");
      }
    }
    return results;
  }

  private List<Sentence> getRelevantSentences(List<String> inputs, String token, int maxSentences) throws IOException {
    List<Sentence> sentences = new ArrayList<>();
    for (String input : inputs) {
      if (new File(input).isDirectory()) {
        File file = new File(input, token + ".txt");
        if (!file.exists()) {
          throw new RuntimeException("File with example sentences not found: " + file);
        }
        try (FileInputStream fis = new FileInputStream(file)) {
          SentenceSource sentenceSource = new PlainTextSentenceSource(fis, language);
          sentences = getSentencesFromSource(inputs, token, maxSentences, sentenceSource);
        }
      } else {
        SentenceSource sentenceSource = MixingSentenceSource.create(inputs, language);
        sentences = getSentencesFromSource(inputs, token, maxSentences, sentenceSource);
      }
    }
    return sentences;
  }

  private List<Sentence> getSentencesFromSource(List<String> inputs, String token, int maxSentences, SentenceSource sentenceSource) {
    List<Sentence> sentences = new ArrayList<>();
    Pattern pattern = Pattern.compile(".*\\b" + (caseSensitive ? token : token.toLowerCase()) + "\\b.*");
    while (sentenceSource.hasNext()) {
      Sentence sentence = sentenceSource.next();

View on GitHub (pinned to 2e990059ce)

Solutions

1. Create or place the missing <token>.txt file inside the input directory with example sentences for the token
2. Verify the token list matches the actual filenames (case-sensitive) in the input directories
3. Check the input directory paths passed on the command line point to the correct language corpus
4. Wrap the run in try-catch and skip tokens without corpus files if partial evaluation is acceptable

Example fix

// before
java org.languagetool.dev.bigdata.ConfusionRuleEvaluator en-US /data/confusion/ pairs.txt
// java: /data/confusion/their/ exists but their.txt missing
// after
ls /data/confusion/their/their.txt || generate_corpus.sh their > /data/confusion/their/their.txt
// then re-run the evaluator

for (String input : inputs) {
  File f = new File(input, token + ".txt");
  if (new File(input).isDirectory() && !f.exists()) {
    throw new IllegalStateException("Pre-check: missing corpus file " + f);
  }
}

When it happens

Trigger: Running ConfusionRuleEvaluator with input directories that lack a <token>.txt file for one of the confusion-set words; token name mismatch in filename (case, spelling, POS suffix); pointing at a directory for the wrong language corpus.

Common situations: Partial corpus downloads; confusion sets generated for a different language than the downloaded data; tokens containing characters that were sanitized out of filenames; typos in the token list.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• File with example sentences not found:
• basePath does not exist:
• basePath does not exist:
• basePath does not exist:
• basePath does not exist:

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/22fae483013b99b7. Report an issue: GitHub.