languagetool-org/languagetool · error

No hits for

Error message

No hits for 

What it means

StartTokenCounter counts occurrences of sentence-start tokens in an ngram Lucene index. For every term looked up via searcher.search(new TermQuery(new Term("ngram", term)), 3), it requires at least one hit; a totalHits of 0 means the term is absent from the index, so counting cannot proceed and it throws.

Source

Thrown at languagetool-dev/src/main/java/org/languagetool/dev/archive/StartTokenCounter.java:64

    File dir = new File("/data/google-ngram-index/en/2grams");
    try (FSDirectory directory = FSDirectory.open(dir.toPath());
         IndexReader reader = DirectoryReader.open(directory)) {
      IndexSearcher searcher = new IndexSearcher(reader);
      Fields fields = MultiFields.getFields(reader);
      Terms ngrams = fields.terms("ngram");
      TermsEnum iterator = ngrams.iterator();
      BytesRef next;
      int i = 0;
      while ((next = iterator.next()) != null) {
        String term = next.utf8ToString();
        if (term.startsWith(LanguageModel.GOOGLE_SENTENCE_START)) {
          if (term.matches(".*_(ADJ|ADV|NUM|VERB|ADP|NOUN|PRON|CONJ|DET|PRT)$")) {
            //System.out.println("ignore: " + term);
            continue;
          }
          TopDocs topDocs = searcher.search(new TermQuery(new Term("ngram", term)), 3);
          if (topDocs.totalHits == 0) {
            throw new RuntimeException("No hits for " + term + ": " + topDocs.totalHits);
          } else if (topDocs.totalHits == 1) {
            int docId = topDocs.scoreDocs[0].doc;
            Document document = reader.document(docId);
            Long count = Long.parseLong(document.get("count"));
            //System.out.println(term + " -> " + count);
            totalCount += count;
            if (++i % 10_000 == 0) {
              System.out.println(i + " ... " + totalCount);
            }
          } else {
            throw new RuntimeException("More hits than expected for " + term + ": " + topDocs.totalHits);
          }
        }
      }
    }
    System.out.println("==> " + totalCount);
  }
  

View on GitHub (pinned to 2e990059ce)

Solutions

1. Confirm the Lucene index was built completely and from the same corpus the terms come from; rebuild with AggregatedNgramToLucene if needed.
2. Log and skip unknown terms instead of failing: change the throw to a warning + continue for rare OOV tokens.
3. Check that the query field name "ngram" and term formatting (including any _POS suffix) exactly match what getDoc() indexed.
4. Open the index with Luke or a small probe program to verify the missing term really is absent.

Example fix

// before
TopDocs topDocs = searcher.search(new TermQuery(new Term("ngram", term)), 3);
if (topDocs.totalHits == 0) {
  throw new RuntimeException("No hits for " + term + ": " + topDocs.totalHits);
}
// after
TopDocs topDocs = searcher.search(new TermQuery(new Term("ngram", term)), 3);
if (topDocs.totalHits == 0) {
  System.err.println("WARN: no index hits, skipping term: " + term);
  continue;
}

// probe the index before batch counting
try (IndexReader reader = DirectoryReader.open(indexDir)) {
  IndexSearcher s = new IndexSearcher(reader);
  if (s.search(new TermQuery(new Term("ngram", term)), 1).totalHits == 0) {
    System.err.println("Term not in index: " + term);
  }
}

try {
  TopDocs topDocs = searcher.search(new TermQuery(new Term("ngram", term)), 3);
  if (topDocs.totalHits == 0) { /* log OOV and continue */ }
} catch (RuntimeException e) {
  skippedTerms.add(term);
}

When it happens

Trigger: Querying the Lucene ngram index with a term that was never indexed — e.g. the input corpus (parsed TSV, POS-stripped) contains a token whose exact surface form with the trailing _POS suffix does not exist in the index, the index was built from a different/smaller corpus, or the field name "ngram" doesn't match the indexed field.

Common situations: Building the Lucene index from one ngram corpus (e.g. Google Books) but querying with terms from another (web); index build interrupted so documents are missing; casing or morphological suffix mismatch between query term and indexed ngram strings.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• More hits than expected for
• No ngram data found for:
• Got more than one hit for:
• Directory must contain at least '1grams', '2grams', and '3gr
• Expected at least '1grams', '2grams', and '3grams' sub direc

AI-assisted analysis of languagetool-org/languagetool@2e990059ce (2026-09-06). Data as JSON: /api/errors/6f65560b99791ddb. Report an issue: GitHub.