{"record":{"id":"22fae483013b99b7","repo":"languagetool-org/languagetool","slug":"file-with-example-sentences-not-found","errorCode":null,"errorMessage":"File with example sentences not found: ","messagePattern":"File with example sentences not found: ","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"languagetool-dev/src/main/java/org/languagetool/dev/bigdata/ConfusionRuleEvaluator.java","lineNumber":214,"sourceCode":"        //System.out.printf(ENGLISH, \"Precision:    %.3f (%d false positives)\\n\", precision, evalValues.falsePositives);\n        //System.out.printf(ENGLISH, \"Recall:       %.3f (%d false negatives)\\n\", recall, evalValues.falseNegatives);\n        //double fMeasure = FMeasure.getWeightedFMeasure(precision, recall);\n        //System.out.printf(ENGLISH, \"F-measure:    %.3f (beta=0.5)\\n\", fMeasure);\n        //System.out.printf(ENGLISH, \"Good Matches: %d (true positives)\\n\", evalValues.truePositives);\n        //System.out.printf(ENGLISH, \"All matches:  %d\\n\", evalValues.truePositives + evalValues.falsePositives);\n        System.out.printf(summary + \"\\n\");\n      }\n    }\n    return results;\n  }\n\n  private List<Sentence> getRelevantSentences(List<String> inputs, String token, int maxSentences) throws IOException {\n    List<Sentence> sentences = new ArrayList<>();\n    for (String input : inputs) {\n      if (new File(input).isDirectory()) {\n        File file = new File(input, token + \".txt\");\n        if (!file.exists()) {\n          throw new RuntimeException(\"File with example sentences not found: \" + file);\n        }\n        try (FileInputStream fis = new FileInputStream(file)) {\n          SentenceSource sentenceSource = new PlainTextSentenceSource(fis, language);\n          sentences = getSentencesFromSource(inputs, token, maxSentences, sentenceSource);\n        }\n      } else {\n        SentenceSource sentenceSource = MixingSentenceSource.create(inputs, language);\n        sentences = getSentencesFromSource(inputs, token, maxSentences, sentenceSource);\n      }\n    }\n    return sentences;\n  }\n\n  private List<Sentence> getSentencesFromSource(List<String> inputs, String token, int maxSentences, SentenceSource sentenceSource) {\n    List<Sentence> sentences = new ArrayList<>();\n    Pattern pattern = Pattern.compile(\".*\\\\b\" + (caseSensitive ? token : token.toLowerCase()) + \"\\\\b.*\");\n    while (sentenceSource.hasNext()) {\n      Sentence sentence = sentenceSource.next();","sourceCodeStart":196,"sourceCodeEnd":232,"githubUrl":"https://github.com/languagetool-org/languagetool/blob/2e990059ce67d5e2a0f7f7ca5d31160c6709df4b/languagetool-dev/src/main/java/org/languagetool/dev/bigdata/ConfusionRuleEvaluator.java#L196-L232","documentation":"ConfusionRuleEvaluator.getRelevantSentences expects each input directory to contain a per-token corpus file named <token>.txt. When the directory exists but the file for the current confusion token is missing, it throws this RuntimeException. It is a dev-tooling corpus-preparation check: without the example sentence file, no evaluation data can be gathered for that token.","triggerScenarios":"Running ConfusionRuleEvaluator with input directories that lack a <token>.txt file for one of the confusion-set words; token name mismatch in filename (case, spelling, POS suffix); pointing at a directory for the wrong language corpus.","commonSituations":"Partial corpus downloads; confusion sets generated for a different language than the downloaded data; tokens containing characters that were sanitized out of filenames; typos in the token list.","solutions":["Create or place the missing <token>.txt file inside the input directory with example sentences for the token","Verify the token list matches the actual filenames (case-sensitive) in the input directories","Check the input directory paths passed on the command line point to the correct language corpus","Wrap the run in try-catch and skip tokens without corpus files if partial evaluation is acceptable"],"exampleFix":"// before\njava org.languagetool.dev.bigdata.ConfusionRuleEvaluator en-US /data/confusion/ pairs.txt\n// java: /data/confusion/their/ exists but their.txt missing\n// after\nls /data/confusion/their/their.txt || generate_corpus.sh their > /data/confusion/their/their.txt\n// then re-run the evaluator","handlingStrategy":"validation","validationCode":"for (String input : inputs) {\n  File f = new File(input, token + \".txt\");\n  if (new File(input).isDirectory() && !f.exists()) {\n    throw new IllegalStateException(\"Pre-check: missing corpus file \" + f);\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Generate corpus files for every token in the confusion set before evaluation","Script a pre-flight check that iterates tokens and asserts each <token>.txt exists","Keep token lists and corpus filenames generated from the same source"],"tags":["file-not-found","dev-tooling","corpus"],"backgroundTag":"file-not-found","analyzedSha":"2e990059ce67d5e2a0f7f7ca5d31160c6709df4b","analyzedAt":"2026-09-06T09:20:17.015Z","contentChangedAt":"2026-09-06T09:20:17.015Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}