languagetool-org/languagetool · error

Invalid confusion set entry:

Error message

Invalid confusion set entry: 

What it means

ProhibitedCompoundRuleEvaluator.main iterates the loaded confusion set and validates that each ConfusionPair has at least two terms; a pair with fewer than 2 words is thrown as 'Invalid confusion set entry: <pair>'. The evaluator needs exactly a word pair to evaluate confusion.

Source

Thrown at languagetool-dev/src/main/java/org/languagetool/dev/bigdata/ProhibitedCompoundRuleEvaluator.java:287

    String confusionSetFile = args[0];
    String langCode = args[1];
    Language lang = Languages.getLanguageForShortCode(langCode);
    ConfusionSetLoader loader = new ConfusionSetLoader(lang);
    Map<String, List<ConfusionPair>> confusionSet = loader.loadConfusionPairs(new FileInputStream(confusionSetFile));
    LanguageModel languageModel = new LuceneLanguageModel(new File(args[2], lang.getShortCode()));
    //LanguageModel languageModel = new BerkeleyRawLanguageModel(new File("/media/Data/berkeleylm/google_books_binaries/ger.blm.gz"));
    //LanguageModel languageModel = new BerkeleyLanguageModel(new File("/media/Data/berkeleylm/google_books_binaries/ger.blm.gz"));
    List<String> inputsFiles = new ArrayList<>();
    inputsFiles.add(args[3]);
    if (args.length >= 5) {
      inputsFiles.add(args[4]);
    }
    ProhibitedCompoundRuleEvaluator generator = new ProhibitedCompoundRuleEvaluator(lang, languageModel);
    for (List<ConfusionPair> entries : confusionSet.values()) {
      for (ConfusionPair pair : entries) {
          ConfusionString[] words  = pair.getTerms().toArray(new ConfusionString[0]);
          if (words.length < 2) {
            throw new RuntimeException("Invalid confusion set entry: " + pair);
          }
          generator.run(inputsFiles, words[0].getString(), words[1].getString(), MAX_SENTENCES, EVAL_FACTORS);
      }
    }
    long endTime = System.currentTimeMillis();
    System.out.println("\nTime: " + (endTime-startTime)+"ms");
  }
}

View on GitHub (pinned to 2e990059ce)

Solutions

1. Fix the confusion set file so every pair contains at least two ConfusionString terms
2. Add a pre-validation pass over the confusion set to report and drop short entries
3. Regenerate the confusion set from source data if entries got corrupted

Example fix

// before (confusion set XML)
<pair><w>Haus</w></pair> <!-- only one term -->
// after
<pair><w>Haus</w><w>Bau</w></pair>

for (List<ConfusionPair> entries : confusionSet.values()) {
  for (ConfusionPair pair : entries) {
    if (pair.getTerms().size() < 2) {
      System.err.println("Skipping short pair: " + pair);
    }
  }
}

try {
  ProhibitedCompoundRuleEvaluator.main(args);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Invalid confusion set entry")) {
    System.err.println("Fix confusion set: " + e.getMessage());
  }
}

When it happens

Trigger: A confusion-set XML/entry defining a pair with only one (or zero) terms, making words.length < 2; malformed confusion-set file entries loaded at startup.

Common situations: Hand-edited confusion set files with truncated entries; parser accepting single-term pairs; merging word lists that dropped one side of a pair.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• No rules are active. Please make sure your rule ids (<option
• factor must be > 0: " + score
• Probability must be >= 0:
• <antipattern>s can only contain <example>s without errors (i
• 'skip' should be between -1 and ${Byte.MAX_VALUE}

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/f4d6736872134a39. Report an issue: GitHub.