languagetool-org/languagetool · error

No ngram data found for:

Error message

No ngram data found for:

What it means

AggregatedNgramToLucene.indexLine() splits each input line into an ngram and its count, then looks up a LuceneIndex keyed by the ngram's token count (word count). If indexes.get(ngramParts.length) returns null — the aggregated file contains an ngram length for which no index was opened — the line cannot be routed and the tool throws with the offending line parts.

Solutions

  1. Ensure the indexes map is initialized for every ngram length present in the input (check the tool's ngram size configuration/main arguments).
  2. Pre-validate the aggregated input: filter or split files by ngram length before indexing.
  3. Log and skip unindexed lengths instead of throwing if out-of-range lines are expected noise.
  4. Inspect the offending line printed in Arrays.toString(lineParts) to check whether it's malformed rather than a genuine long ngram.

Example fix

// before
LuceneIndex index = indexes.get(ngramParts.length);
if (index == null) {
  throw new RuntimeException("No ngram data found for: " + Arrays.toString(lineParts));
}
// after
LuceneIndex index = indexes.get(ngramParts.length);
if (index == null) {
  System.err.println("WARN: no index for ngram length " + ngramParts.length + ", skipping: " + ngram);
  return;
}
Defensive patterns

Strategy: validation

Validate before calling

// verify ngram lengths in the input before indexing
int maxLen = indexes.keySet().stream().max(Integer::compare).orElse(0);
for (String line : Files.readAllLines(input)) {
  int len = line.split(" ")[0].trim().split(" ").length;
  if (len < 1 || len > maxLen) {
    System.err.println("Out-of-range ngram length " + len + ": " + line);
  }
}

Try / catch

try {
  indexer.indexInputFile(path);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("No ngram data found")) {
    System.err.println("Re-run with indexes covering all ngram lengths in input");
  }
}

Prevention

When it happens

Trigger: Processing an aggregated ngram TSV whose lines contain ngrams longer (or shorter) than the configured set of indexes — e.g. the tool was initialized for 1-5 word ngrams but the input contains 6-grams; also a malformed line where the split produces an unexpected token count.

Common situations: Feeding a mixed or misconfigured aggregated corpus into the indexer; running the tool with a max-ngram-size setting smaller than the data; whitespace/format changes in the aggregation output shifting the field layout.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/efe20ad95d1c8bb4. Report an issue: GitHub.

Appendix: source

Thrown at languagetool-dev/src/main/java/org/languagetool/dev/bigdata/AggregatedNgramToLucene.java:80

        indexLine(line);
      }
    }
  }

  private void indexLine(String line) throws IOException {
    if (lineCount++ % 250_000 == 0) {
      System.out.printf(Locale.ENGLISH, "Indexing line %d\n", lineCount);
    }
    String[] lineParts = line.split("\t");
    if (lineParts.length != 2) {
      System.err.println("Not 2 parts but " + lineParts.length + ", ignoring: '" + line + "'");
      return;
    }
    String ngram = lineParts[0];
    String[] ngramParts = ngram.split(" ");
    LuceneIndex index = indexes.get(ngramParts.length);
    if (index == null) {
      throw new RuntimeException("No ngram data found for: " + Arrays.toString(lineParts));
    }
    long count = Long.parseLong(lineParts[1]);
    if (ngramParts.length == 1) {
      totalTokenCount += count;
    }
    index.indexWriter.addDocument(getDoc(ngram, count));
  }

  @NotNull
  private Document getDoc(String ngram, long count) {
    Document doc = new Document();
    doc.add(new Field("ngram", ngram, StringField.TYPE_NOT_STORED));  // use StringField.TYPE_STORED for easier debugging with e.g. Luke
    doc.add(getCountField(count));
    return doc;
  }

  @NotNull
  private LongField getCountField(long count) {

View on GitHub (pinned to 2e990059ce)