{"record":{"id":"8d05033072790f4e","repo":"languagetool-org/languagetool","slug":"elements-with-unified-tokens-are-not-supported","errorCode":null,"errorMessage":"Elements with unified tokens are not supported.","messagePattern":"Elements with unified tokens are not supported\\.","errorType":"exception","errorClass":"UnsupportedPatternRuleException","httpStatus":null,"severity":"error","filePath":"languagetool-wikipedia/src/main/java/org/languagetool/dev/index/PatternRuleQueryBuilder.java","lineNumber":249,"sourceCode":"      if (str.contains(\"?iu\") || str.contains(\"?-i\")) {\n        // Lucene's RegexpQuery doesn't seem to support this\n        throw new UnsupportedPatternRuleException(\"Regex constructs like '?iu' and '?-i' are not supported: \" + patternToken);\n      }\n      return new RegexpQuery(term);\n    } catch (IllegalArgumentException e) {\n      // constructs like \"\\p{Punct}\" not supported by Lucene RegExp:\n      throw new UnsupportedPatternRuleException(\"Advanced regex like '\\\\p{Punct}' are not supported: \" + patternToken);\n    }\n  }\n\n  private void checkUnsupportedElement(PatternToken patternPatternToken)\n      throws UnsupportedPatternRuleException {\n    if (patternPatternToken.hasOrGroup()) {\n      // TODO: this is not enough: the first of the tokens in the <or> group will not get into this branch\n      throw new UnsupportedPatternRuleException(\"<or> not yet supported.\");\n    }\n    if (patternPatternToken.isUnified()) {\n      throw new UnsupportedPatternRuleException(\"Elements with unified tokens are not supported.\");\n    }\n    if (patternPatternToken.getString() != null && patternPatternToken.getString().matches(\"\\\\\\\\\\\\d+\")) { // e.g. \"\\1\"\n      throw new UnsupportedPatternRuleException(\"Elements with only match references (e.g. \\\\1) are not supported.\");\n    }\n  }\n\n}\n","sourceCodeStart":231,"sourceCodeEnd":257,"githubUrl":"https://github.com/languagetool-org/languagetool/blob/2e990059ce67d5e2a0f7f7ca5d31160c6709df4b/languagetool-wikipedia/src/main/java/org/languagetool/dev/index/PatternRuleQueryBuilder.java#L231-L257","documentation":"Search-index guard in checkUnsupportedElement: the pattern token uses unified (matching) tokens, whose cross-token agreement semantics cannot be expressed in a Lucene query.","triggerScenarios":"Thrown at languagetool-wikipedia/src/main/java/org/languagetool/dev/index/PatternRuleQueryBuilder.java:249 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Remove unification from the element for indexing purposes","Skip the rule during index-based checking","Post-filter matches with the full rule engine after a relaxed index query"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"2e990059ce67d5e2a0f7f7ca5d31160c6709df4b","analyzedAt":"2026-09-06T09:20:17.015Z","contentChangedAt":"2026-09-06T09:20:17.015Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}