TencentCloud/TencentDB-Agent-Memory · warning · EmbeddingNotReadyError

Local embedding model is still loading…

Error message

Local embedding model is still loading (download/initialization in progress). Please try again later.

What it means

assertReady() throws EmbeddingNotReadyError when initState is 'initializing', meaning startWarmup() has been called but the model is still downloading/initializing. The embedder deliberately does not queue or block: embed()/embedBatch() fail fast so the caller can retry later or fall back. The context is not yet set, so serving requests now would be impossible.

Solutions

Retry the embed call after a delay until initState becomes 'ready' (poll or exponential backoff).
Await the warmup promise before issuing embed calls, e.g. expose/track startWarmup()'s returned promise and gate requests on it.
Add a readiness gate (health check / lazy init in a request queue) so requests wait for model readiness instead of failing.
Fall back to a remote embedding API during local model warmup.

Example fix

// before
await embedder.startWarmup();
embedder.embed(text); // not awaited; may hit 'still loading'
// after
await embedder.startWarmup(); // wait until model is ready
const vec = await embedder.embed(text);

Defensive patterns

Strategy: retry

Validate before calling

if ((embedder as any).initState === "initializing") {
  await waitForReady(embedder); // poll/backoff until initState === "ready"
}

Type guard

function isEmbeddingNotReadyError(e: unknown): e is EmbeddingNotReadyError {
  return e instanceof EmbeddingNotReadyError;
}

Try / catch

async function embedWithRetry(embedder, text, tries = 10) {
  for (let i = 0; i < tries; i++) {
    try {
      return await embedder.embed(text);
    } catch (e) {
      if (isEmbeddingNotReadyError(e) && /still loading/.test(e.message)) {
        await new Promise(r => setTimeout(r, 500 * (i + 1)));
        continue;
      }
      throw e;
    }
  }
  throw new Error("embedding model did not become ready in time");
}

Prevention

Gate request handling on warmup completion (health-check readiness or awaited startWarmup promise).
Never fire embed calls concurrently with startWarmup in startup code; await it.
Warm up the model before the load balancer marks the instance healthy.
Use exponential backoff with a cap when retrying while-loading errors.

When it happens

Trigger: Calling embed() or embedBatch() concurrently with an in-flight startWarmup() — e.g. the first request after server boot while the GGUF model is still downloading, or right after close()+startWarmup() with a large model still loading.

Common situations: Cold-start race: traffic arrives before the model finishes its first download/load (slow network, multi-GB GGUF); tests that don't await warmup before calling embed; load balancer sending traffic to a just-started instance.

Related errors

AI-assisted analysis of TencentCloud/TencentDB-Agent-Memory@3efcd317b8 (2026-09-01). Data as JSON: /api/errors/0fb00f4e0fbf7617. Report an issue: GitHub.

Appendix: source

Thrown at MemoryCore/src/core/store/embedding.ts:290

      this.logger?.info(`${TAG} Local embedding resources released`);
    }
  }

  /**
   * Assert the model is ready. Throws EmbeddingNotReadyError if not.
   */
  private assertReady(): void {
    if (this.initState === "ready" && this.embeddingContext) {
      return;
    }
    if (this.initState === "failed") {
      throw new EmbeddingNotReadyError(
        `Local embedding model initialization failed: ${this.initError?.message ?? "unknown error"}. ` +
        `Call startWarmup() to retry.`,
      );
    }
    if (this.initState === "initializing") {
      throw new EmbeddingNotReadyError(
        "Local embedding model is still loading (download/initialization in progress). Please try again later.",
      );
    }
    // "idle" — startWarmup() was never called
    throw new EmbeddingNotReadyError(
      "Local embedding model warmup has not been started. Call startWarmup() first.",
    );
  }

  /**
   * Truncate input text to stay within the model's context window.
   * embeddinggemma-300m has a 256-token limit; we use a character-based
   * heuristic (LOCAL_MAX_INPUT_CHARS) as a safe proxy.
   */
  private truncateInput(text: string): string {
    if (text.length <= LOCAL_MAX_INPUT_CHARS) return text;
    this.logger?.debug?.(
      `${TAG} Input truncated from ${text.length} to ${LOCAL_MAX_INPUT_CHARS} chars (model context limit)`,

View on GitHub (pinned to 3efcd317b8)