{"record":{"id":"ba3634386103ed4a","repo":"tobi/qmd","slug":"reranker-unavailable-skipping-reranking-detai","errorCode":null,"errorMessage":"Reranker unavailable — skipping reranking (${detail}). Use --no-rerank to silence this warning.","messagePattern":"Reranker unavailable — skipping reranking \\((.+?)\\)\\. Use --no-rerank to silence this warning\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/llm.ts","lineNumber":1353,"sourceCode":"    this.rerankContextsCreatePromise = (async () => {\n      this.touchActivity();\n      const model = await this.ensureRerankModel();\n      const n = Math.min(await this.computeParallelism(1000), 4);\n      const threads = await this.threadsPerContext(n);\n      for (let i = 0; i < n; i++) {\n        try {\n          this.rerankContexts.push(await model.createRankingContext({\n            contextSize: LlamaCpp.RERANK_CONTEXT_SIZE,\n            ...(threads > 0 ? { threads } : {}),\n          }));\n        } catch (error) {\n          if (this.rerankContexts.length === 0) {\n            // Surface the underlying failure (e.g. out of VRAM). A previous\n            // \"retry without flash attention\" path was dead: ranking contexts\n            // never accepted that option, so the retry repeated identical\n            // arguments and the real error was discarded.\n            const detail = error instanceof Error ? error.message : String(error);\n            console.warn(\n              `Reranker unavailable — skipping reranking (${detail}). ` +\n              \"Use --no-rerank to silence this warning.\",\n            );\n            return [];\n          }\n          // At least one context exists — continue with reduced parallelism.\n          break;\n        }\n      }\n      this.touchActivity();\n      return this.rerankContexts;\n    })();\n\n    try {\n      return await this.rerankContextsCreatePromise;\n    } finally {\n      this.rerankContextsCreatePromise = null;\n    }","sourceCodeStart":1335,"sourceCodeEnd":1371,"githubUrl":"https://github.com/tobi/qmd/blob/dbfd0b4736aeaf761d1a16ca8e424f071df8feb9/src/llm.ts#L1335-L1371","documentation":"During query-time reranking, if loading/running the qwen3-reranker model fails and there are no fallback ranking contexts, the LLM layer gives up on reranking entirely: it warns 'Reranker unavailable — skipping reranking ({detail}). Use --no-rerank to silence this warning.' and returns an empty ranking, so results come out in their pre-rerank (RRF) order. The detail string surfaces the underlying failure (e.g. out of VRAM).","triggerScenarios":"Running `qmd query` (which reranks by default) on a machine where the reranker model cannot load or run: insufficient VRAM/RAM, a broken or missing model download, an incompatible llama.cpp build, or GPU driver issues. The catch notices rerankContexts.length === 0, logs the warning, and returns [].","commonSituations":"First run on a low-memory machine before models finished downloading; another process consuming GPU memory; running in a container without GPU access; node-llama-cpp native binary mismatch after a Bun/dependency upgrade; old hardware without flash-attention support.","solutions":["Read the parenthesized detail: for 'out of VRAM'/'out of memory', free GPU memory or close other model-using processes, then retry","Silence the warning and accept RRF-only ranking with `qmd query '...' --no-rerank` (or make it permanent in config) when reranking quality is not required","Verify the model setup with `qmd doctor` — it diagnoses model, device, and index issues including download problems","Reinstall/refresh node-llama-cpp and its model assets (`bun install`) if the native layer is broken after an upgrade"],"exampleFix":"# before\nqmd query 'deployment notes'   # warns: Reranker unavailable — skipping reranking\n\n# after\nqmd query 'deployment notes' --no-rerank\n# or fix the environment first:\nqmd doctor","handlingStrategy":"fallback","validationCode":"// CLI-level: decide before running whether to attempt reranking\nimport { execSync } from 'node:child_process';\n\nfunction hasGpuHeadroom(): boolean {\n  try {\n    execSync('nvidia-smi --query-gpu=memory.free --format=csv,noheader,nounits');\n    return true;\n  } catch { return false; }\n}\nconst args = hasGpuHeadroom() ? ['query', q] : ['query', q, '--no-rerank'];","typeGuard":"function shouldAttemptRerank(freeVramMb: number | null, modelLoaded: boolean): boolean {\n  return modelLoaded && (freeVramMb === null || freeVramMb > 1500);\n}","tryCatchPattern":null,"preventionTips":["Run qmd doctor before first use to validate model and device","Use --no-rerank on memory-constrained machines or CI; RRF results are still good","Close other GPU-consuming processes (other llama.cpp instances, games, notebooks) before querying","After dependency upgrades, reinstall node-llama-cpp assets to avoid native mismatches"],"tags":["llm","reranker","vram","node-llama-cpp","fallback","model-loading"],"backgroundTag":"model-load-out-of-memory","analyzedSha":"dbfd0b4736aeaf761d1a16ca8e424f071df8feb9","analyzedAt":"2026-08-28T18:07:46.628Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}