tobi/qmd · warning

Reranker unavailable — skipping reranking (${detail}). Use -

Error message

Reranker unavailable — skipping reranking (${detail}). Use --no-rerank to silence this warning.

What it means

During query-time reranking, if loading/running the qwen3-reranker model fails and there are no fallback ranking contexts, the LLM layer gives up on reranking entirely: it warns 'Reranker unavailable — skipping reranking ({detail}). Use --no-rerank to silence this warning.' and returns an empty ranking, so results come out in their pre-rerank (RRF) order. The detail string surfaces the underlying failure (e.g. out of VRAM).

Source

Thrown at src/llm.ts:1353

    this.rerankContextsCreatePromise = (async () => {
      this.touchActivity();
      const model = await this.ensureRerankModel();
      const n = Math.min(await this.computeParallelism(1000), 4);
      const threads = await this.threadsPerContext(n);
      for (let i = 0; i < n; i++) {
        try {
          this.rerankContexts.push(await model.createRankingContext({
            contextSize: LlamaCpp.RERANK_CONTEXT_SIZE,
            ...(threads > 0 ? { threads } : {}),
          }));
        } catch (error) {
          if (this.rerankContexts.length === 0) {
            // Surface the underlying failure (e.g. out of VRAM). A previous
            // "retry without flash attention" path was dead: ranking contexts
            // never accepted that option, so the retry repeated identical
            // arguments and the real error was discarded.
            const detail = error instanceof Error ? error.message : String(error);
            console.warn(
              `Reranker unavailable — skipping reranking (${detail}). ` +
              "Use --no-rerank to silence this warning.",
            );
            return [];
          }
          // At least one context exists — continue with reduced parallelism.
          break;
        }
      }
      this.touchActivity();
      return this.rerankContexts;
    })();

    try {
      return await this.rerankContextsCreatePromise;
    } finally {
      this.rerankContextsCreatePromise = null;
    }

View on GitHub (pinned to dbfd0b4736)

Solutions

1. Read the parenthesized detail: for 'out of VRAM'/'out of memory', free GPU memory or close other model-using processes, then retry
2. Silence the warning and accept RRF-only ranking with `qmd query '...' --no-rerank` (or make it permanent in config) when reranking quality is not required
3. Verify the model setup with `qmd doctor` — it diagnoses model, device, and index issues including download problems
4. Reinstall/refresh node-llama-cpp and its model assets (`bun install`) if the native layer is broken after an upgrade

Example fix

# before
qmd query 'deployment notes'   # warns: Reranker unavailable — skipping reranking

# after
qmd query 'deployment notes' --no-rerank
# or fix the environment first:
qmd doctor

// CLI-level: decide before running whether to attempt reranking
import { execSync } from 'node:child_process';

function hasGpuHeadroom(): boolean {
  try {
    execSync('nvidia-smi --query-gpu=memory.free --format=csv,noheader,nounits');
    return true;
  } catch { return false; }
}
const args = hasGpuHeadroom() ? ['query', q] : ['query', q, '--no-rerank'];

function shouldAttemptRerank(freeVramMb: number | null, modelLoaded: boolean): boolean {
  return modelLoaded && (freeVramMb === null || freeVramMb > 1500);
}

When it happens

Trigger: Running `qmd query` (which reranks by default) on a machine where the reranker model cannot load or run: insufficient VRAM/RAM, a broken or missing model download, an incompatible llama.cpp build, or GPU driver issues. The catch notices rerankContexts.length === 0, logs the warning, and returns [].

Common situations: First run on a low-memory machine before models finished downloading; another process consuming GPU memory; running in a container without GPU access; node-llama-cpp native binary mismatch after a Bun/dependency upgrade; old hardware without flash-attention support.

Related errors

• Failed to create any embedding context
• ⚠ Error rate too high (${activeErrorCount()}/${processed}) —
• LLM operations are disabled in CI (set CI=true)
• Failed to get embedding dimensions from first chunk
• [qmd] AST grammar unavailable for ${language}: ${message}

AI-assisted analysis of tobi/qmd@dbfd0b4736 (2026-08-28). Data as JSON: /api/errors/ba3634386103ed4a. Report an issue: GitHub.