What causes "Searcher for ngram size ${ngramSize} already exists" in languagetool-org/languagetool?

Remove the duplicate addIndex call for that ngram size

languagetool-org/languagetool · warning · RuntimeException

Searcher for ngram size ${ngramSize} already exists

Error message

Searcher for ngram size ${ngramSize} already exists

What it means

The private addIndex() method refuses to register a Lucene searcher for an ngram size that already has one in luceneSearcherMap, throwing RuntimeException. It is an internal-consistency guard against adding the same ngram index twice.

Source

Thrown at languagetool-core/src/main/java/org/languagetool/languagemodel/LuceneSingleIndexLanguageModel.java:116

      throw new RuntimeException("No directories '1grams' ... '3grams' found in " + topIndexDir);
    }
    maxNgram = Collections.<Integer>max(luceneSearcherMap.keySet());
  }

  public LuceneSingleIndexLanguageModel(int maxNgram) {
    this.maxNgram = maxNgram;
    this.topIndexDir = null;
  }

  protected void doValidateDirectory(File topIndexDir) {
    validateDirectory(topIndexDir);
  }

  private void addIndex(File topIndexDir, int ngramSize) {
    File indexDir = new File(topIndexDir, ngramSize + "grams");
    if (indexDir.exists() && indexDir.isDirectory()) {
      if (luceneSearcherMap.containsKey(ngramSize)) {
        throw new RuntimeException("Searcher for ngram size " + ngramSize + " already exists");
      }
      luceneSearcherMap.put(ngramSize, getCachedLuceneSearcher(indexDir));
      indexes.add(indexDir);
    }
  }

  @Override
  public long getCount(List<String> tokens) {
    if (tokens.size() > maxNgram) {
      throw new RuntimeException("Requested " + tokens.size() + "gram but index has only up to " + maxNgram + "gram: " + tokens);
    }
    Objects.requireNonNull(tokens);
    Term term = new Term("ngram", String.join(" ", tokens));
    return getCount(term, getLuceneSearcher(tokens.size()));
  }

  @Override
  public long getCount(String token1) {

View on GitHub (pinned to 2e990059ce)

Solutions

Remove the duplicate addIndex call for that ngram size
Check luceneSearcherMap (or a containsKey guard) before adding an index
Use the public constructors instead of manipulating internal state

Example fix

// before
addIndex(dir, 3);
addIndex(dir, 3); // RuntimeException
// after
if (!luceneSearcherMap.containsKey(3)) addIndex(dir, 3);

Defensive patterns

Strategy: validation

Validate before calling

if (searcherMap.containsKey(ngramSize)) return searcherMap.get(ngramSize); // reuse instead of re-adding

Try / catch

try { addIndex(dir, size); } catch (RuntimeException e) { if (e.getMessage().contains("already exists")) return; throw e; }

Prevention

Treat addIndex as idempotent-required: guard with containsKey before adding
Avoid reflection/subclassing into private internals; use public constructors
Keep track of registered ngram sizes in your own setup code

When it happens

Trigger: Calling addIndex() twice with the same ngramSize for the same model — only possible via internal/reflection code paths, since addIndex is private and the public constructor calls it once per size 1-4.

Common situations: Custom subclassing or reflection-based code that re-invokes addIndex; modifications to the constructor that add the same size twice.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/36f055a92f2814e5. Report an issue: GitHub.