elastic/elasticsearch · error · IllegalArgumentException

Token filter [{name()}] cannot be used to parse synonyms

Error message

Token filter [{name()}] cannot be used to parse synonyms

What it means

WordDelimiterTokenFilterFactory.getSynonymFilter() is hard-coded to throw IllegalArgumentException, identical in spirit to the graph variant. The non-graph word-delimiter filter splits tokens in ways that break the parsing of synonym rules, so Elasticsearch refuses to use it (or any analyzer chain that resolves to it) as the synonym-parsing analyzer.

Source

Thrown at modules/analysis-common/src/main/java/org/elasticsearch/analysis/common/WordDelimiterTokenFilterFactory.java:96

        flags |= getFlag(PRESERVE_ORIGINAL, settings, "preserve_original", false);
        // 1, causes "j2se" to be three tokens; "j" "2" "se"
        flags |= getFlag(SPLIT_ON_NUMERICS, settings, "split_on_numerics", true);
        // If set, causes trailing "'s" to be removed for each subword: "O'Neil's" => "O", "Neil"
        flags |= getFlag(STEM_ENGLISH_POSSESSIVE, settings, "stem_english_possessive", true);
        // If not null is the set of tokens to protect from being delimited
        Set<?> protectedWords = Analysis.getWordSet(env, settings, "protected_words");
        this.protoWords = protectedWords == null ? null : CharArraySet.copy(protectedWords);
        this.flags = flags;
    }

    @Override
    public TokenStream create(TokenStream tokenStream) {
        return new WordDelimiterFilter(tokenStream, charTypeTable, flags, protoWords);
    }

    @Override
    public TokenFilterFactory getSynonymFilter() {
        throw new IllegalArgumentException("Token filter [" + name() + "] cannot be used to parse synonyms");
    }

    public static int getFlag(int flag, Settings settings, String key, boolean defaultValue) {
        if (settings.getAsBoolean(key, defaultValue)) {
            return flag;
        }
        return 0;
    }

    // source => type
    private static final Pattern typePattern = Pattern.compile("(.*)\\s*=>\\s*(.*)\\s*$");

    /**
     * parses a list of MappingCharFilter style rules into a custom byte[] type table
     */
    static byte[] parseTypes(Collection<String> rules) {
        SortedMap<Character, Byte> typeMap = new TreeMap<>();
        for (String rule : rules) {

View on GitHub (pinned to db6a809a66)

Solutions

1. Set an explicit "analyzer" on the synonym/synonym_graph filter that does not include word_delimiter (use 'standard' or 'whitespace').
2. Move word_delimiter into a search-specific analyzer only.
3. Verify with _analyze that the synonym-parsing chain emits one token per synonym term.

Example fix

// before
"my_syn": { "type": "synonym_graph", "synonyms": ["foo,bar"], "analyzer": "my_wd_analyzer" }
// after
"my_syn": { "type": "synonym_graph", "synonyms": ["foo,bar"], "analyzer": "standard" }

// Reject synonym filters that resolve to a word_delimiter analyzer
static String checkSynonymAnalyzer(Map<String,Object> filter, Set<String> wdAnalyzers) {
  if ("synonym".equals(filter.get("type")) || "synonym_graph".equals(filter.get("type"))) {
    Object a = filter.getOrDefault("analyzer", "<default>");
    if (wdAnalyzers.contains(a)) return "analyzer " + a + " contains word_delimiter";
  }
  return null;
}

When it happens

Trigger: A synonym/synonym_graph filter whose effective analyzer chain contains a word_delimiter filter; commonly when 'analyzer' is omitted and the index's default analyzer includes word_delimiter.

Common situations: Reusing a generic analyzer for both indexing and synonym parsing; legacy config that pre-dates the getSynonymFilter guard.

Related errors

• Token filter [{name()}] cannot be used to parse synonyms
• synonym requires either `synonyms`, `synonyms_set` or `synon
• Cannot merge SEARCH_TIME and INDEX_TIME analysis mode.
• failed to build synonyms from [{rules.origin}]
• update_offsets is not supported anymore. Please fix your ana

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/648db2b95a1fef01. Report an issue: GitHub.