What causes "Failed to create any embedding context" in tobi/qmd?

Free memory or reduce embedding context size / threads configuration and retry

tobi/qmd · critical · Error

Failed to create any embedding context

Error message

Failed to create any embedding context

What it means

LlamaCpp failed to create even a single embedding context via model.createEmbeddingContext(). The code retries context creation and only throws when the first attempt fails (embedContexts is empty), meaning node-llama-cpp could not allocate a context window of LlamaCpp.EMBED_CONTEXT_SIZE for the loaded embedding model.

Source

Thrown at src/llm.ts:1218

        try {
          perContextMB = estimateEmbedContextMB({
            modelBytes: statSync(this.embedModelPath).size,
            contextSize: LlamaCpp.EMBED_CONTEXT_SIZE,
          });
        } catch {
          // Keep the baseline if the file cannot be stat'd.
        }
      }
      const n = await this.computeParallelism(perContextMB, EMBED_POOL_RERANK_RESERVE_MB);
      const threads = await this.threadsPerContext(n);
      for (let i = 0; i < n; i++) {
        try {
          this.embedContexts.push(await model.createEmbeddingContext({
            contextSize: LlamaCpp.EMBED_CONTEXT_SIZE,
            ...(threads > 0 ? { threads } : {}),
          }));
        } catch {
          if (this.embedContexts.length === 0) throw new Error("Failed to create any embedding context");
          break;
        }
      }
      this.touchActivity();
      return this.embedContexts;
    })();

    try {
      return await this.embedContextsCreatePromise;
    } finally {
      this.embedContextsCreatePromise = null;
    }
  }

  /**
   * Get a single embed context (for single-embed calls). Uses first from pool.
   */
  private async ensureEmbedContext(): Promise<LlamaEmbeddingContext> {

View on GitHub (pinned to dbfd0b4736)

Solutions

Free memory or reduce embedding context size / threads configuration and retry
Verify the embedding model file downloads correctly and is a valid GGUF (re-download if corrupt)
Run `qmd doctor` to diagnose model/device issues
Check node-llama-cpp native bindings are built for your platform (rebuild after upgrades)

Example fix

// before
const ctxs = await llamaCpp.getEmbedContexts(); // throws if createEmbeddingContext fails
// after
try {
  const ctxs = await llamaCpp.getEmbedContexts();
} catch (e) {
  console.error('Embedding context creation failed — check RAM/VRAM and model file:', e);
  // fall back to BM25-only search: qmd search instead of qmd query/vsearch
}

Defensive patterns

Strategy: fallback

Validate before calling

const free = process.memory?.available ?? require('os').freemem();
if (free < 1_000_000_000) { /* skip embeddings, use BM25 only */ }

Try / catch

try { await llm.embed(text); } catch (e) { if ((e as Error).message.includes('embedding context')) return null; /* degrade to keyword search */ throw e; }

Prevention

Run qmd doctor before embedding workflows
Ensure enough free RAM/VRAM for the embedding model's context size
Keep embedding model files valid and version-matched to node-llama-cpp

When it happens

Trigger: Calling a method that triggers ensureEmbedContexts()/embed() (e.g. embedBatch, qmd embed) when createEmbeddingContext throws — typically insufficient memory/VRAM for the context size, an incompatible or corrupt GGUF embedding model, or the model was disposed/unloaded mid-operation.

Common situations: Low-RAM machines or CI runners where the requested contextSize exceeds available memory; switching to a larger embedding model (embeddinggemma) without enough headroom; GPU layers misconfigured; node-llama-cpp native module build mismatch after upgrade.

Related errors

AI-assisted analysis of tobi/qmd@dbfd0b4736 (2026-08-28). Data as JSON: /api/errors/5ea33391acde79a4. Report an issue: GitHub.