{"record":{"id":"f4d6736872134a39","repo":"languagetool-org/languagetool","slug":"invalid-confusion-set-entry","errorCode":null,"errorMessage":"Invalid confusion set entry: ","messagePattern":"Invalid confusion set entry: ","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"languagetool-dev/src/main/java/org/languagetool/dev/bigdata/ProhibitedCompoundRuleEvaluator.java","lineNumber":287,"sourceCode":"    String confusionSetFile = args[0];\n    String langCode = args[1];\n    Language lang = Languages.getLanguageForShortCode(langCode);\n    ConfusionSetLoader loader = new ConfusionSetLoader(lang);\n    Map<String, List<ConfusionPair>> confusionSet = loader.loadConfusionPairs(new FileInputStream(confusionSetFile));\n    LanguageModel languageModel = new LuceneLanguageModel(new File(args[2], lang.getShortCode()));\n    //LanguageModel languageModel = new BerkeleyRawLanguageModel(new File(\"/media/Data/berkeleylm/google_books_binaries/ger.blm.gz\"));\n    //LanguageModel languageModel = new BerkeleyLanguageModel(new File(\"/media/Data/berkeleylm/google_books_binaries/ger.blm.gz\"));\n    List<String> inputsFiles = new ArrayList<>();\n    inputsFiles.add(args[3]);\n    if (args.length >= 5) {\n      inputsFiles.add(args[4]);\n    }\n    ProhibitedCompoundRuleEvaluator generator = new ProhibitedCompoundRuleEvaluator(lang, languageModel);\n    for (List<ConfusionPair> entries : confusionSet.values()) {\n      for (ConfusionPair pair : entries) {\n          ConfusionString[] words  = pair.getTerms().toArray(new ConfusionString[0]);\n          if (words.length < 2) {\n            throw new RuntimeException(\"Invalid confusion set entry: \" + pair);\n          }\n          generator.run(inputsFiles, words[0].getString(), words[1].getString(), MAX_SENTENCES, EVAL_FACTORS);\n      }\n    }\n    long endTime = System.currentTimeMillis();\n    System.out.println(\"\\nTime: \" + (endTime-startTime)+\"ms\");\n  }\n}\n\n","sourceCodeStart":269,"sourceCodeEnd":297,"githubUrl":"https://github.com/languagetool-org/languagetool/blob/2e990059ce67d5e2a0f7f7ca5d31160c6709df4b/languagetool-dev/src/main/java/org/languagetool/dev/bigdata/ProhibitedCompoundRuleEvaluator.java#L269-L297","documentation":"ProhibitedCompoundRuleEvaluator.main iterates the loaded confusion set and validates that each ConfusionPair has at least two terms; a pair with fewer than 2 words is thrown as 'Invalid confusion set entry: <pair>'. The evaluator needs exactly a word pair to evaluate confusion.","triggerScenarios":"A confusion-set XML/entry defining a pair with only one (or zero) terms, making words.length < 2; malformed confusion-set file entries loaded at startup.","commonSituations":"Hand-edited confusion set files with truncated entries; parser accepting single-term pairs; merging word lists that dropped one side of a pair.","solutions":["Fix the confusion set file so every pair contains at least two ConfusionString terms","Add a pre-validation pass over the confusion set to report and drop short entries","Regenerate the confusion set from source data if entries got corrupted"],"exampleFix":"// before (confusion set XML)\n<pair><w>Haus</w></pair> <!-- only one term -->\n// after\n<pair><w>Haus</w><w>Bau</w></pair>","handlingStrategy":"validation","validationCode":"for (List<ConfusionPair> entries : confusionSet.values()) {\n  for (ConfusionPair pair : entries) {\n    if (pair.getTerms().size() < 2) {\n      System.err.println(\"Skipping short pair: \" + pair);\n    }\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  ProhibitedCompoundRuleEvaluator.main(args);\n} catch (RuntimeException e) {\n  if (e.getMessage().startsWith(\"Invalid confusion set entry\")) {\n    System.err.println(\"Fix confusion set: \" + e.getMessage());\n  }\n}","preventionTips":["Validate confusion-set files (every pair >= 2 terms) before running","Generate confusion sets with a schema-checked tool rather than hand-editing","Report and skip invalid entries instead of aborting the whole run"],"tags":["validation","confusion-set","data-format"],"backgroundTag":"schema-validation-failed","analyzedSha":"2e990059ce67d5e2a0f7f7ca5d31160c6709df4b","analyzedAt":"2026-09-06T09:20:17.015Z","contentChangedAt":"2026-09-06T09:20:17.015Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}