What causes "[agentmemory] Failed to rebuild search index:" in rohitg00/agentmemory?

Inspect the logged `err` — provider HTTP status vs data error

[agentmemory] Failed to rebuild search index:

Error message

[agentmemory] Failed to rebuild search index:

What it means

Warning logged when the fire-and-forget background rebuild (`rebuildIndex(kv)`) rejects. This runs at boot when the BM25 index is empty, iterating every observation and calling the embedding provider per record; any error inside (provider/network failure, corrupted KV records) is caught here so boot is never blocked or crashed. Search stays partially indexed until a successful rebuild.

Source

Thrown at src/index.ts:486

  if (needsRebuild) {
    // Fire-and-forget. rebuildIndex iterates every observation across
    // every session and AWAITS an embedding-provider call per record.
    // On a large corpus + rate-limited embedding endpoint that can
    // take HOURS; awaiting it here blocks every subsequent boot step
    // (including startViewerServer below, leaving the viewer port
    // unbound for the duration). The index lazily fills in over time
    // and search degrades gracefully — partial coverage > no viewer
    // for hours. Errors still surface via the inner .catch.
    void rebuildIndex(kv)
      .then((indexCount) => {
        if (indexCount > 0) {
          bootLog(`Search index rebuilt: ${indexCount} entries`);
          indexPersistence.scheduleSave();
        }
      })
      .catch((err) => {
        console.warn(`[agentmemory] Failed to rebuild search index:`, err);
      });
  } else {
    // Backfill memories into BM25 for users upgrading from <0.9.5: prior
    // versions of mem::remember never indexed memories, so the persisted
    // BM25 covers observations only and `memory_smart_search` returns
    // empty for everything saved via memory_save (#257). Walk KV.memories
    // and add the ones missing from the restored index. Idempotent on
    // re-runs because SearchIndex.has() short-circuits already-indexed
    // ids.
    try {
      const memories = await kv.list<import("./types.js").Memory>(KV.memories);
      let backfilled = 0;
      for (const memory of memories) {
        if (memory.isLatest === false) continue;
        if (!memory.title || !memory.content) continue;
        if (bm25Index.has(memory.id)) continue;
        bm25Index.add({
          id: memory.id,

View on GitHub (pinned to e04ba88819)

Solutions

Inspect the logged `err` — provider HTTP status vs data error
Fix embedding credentials/env (e.g. provider API key, AGENTMEMORY_URL) and restart to retrigger rebuild (it runs whenever BM25 is empty)
For rate limits, reduce corpus or use a provider with higher quota; the index also fills lazily from live traffic
If caused by malformed legacy records, prune/repair those KV entries or upgrade to a version that skips them
Set AGENTMEMORY_DROP_STALE_INDEX=true if stale persisted vectors are involved, then restart

Example fix

// before
[agentmemory] Failed to rebuild search index: Error: 401 Unauthorized from embedding provider
// after
export EMBEDDING_API_KEY=sk-...
$ agentmemory start
Search index rebuilt: 1204 entries

Defensive patterns

Strategy: try-catch

Validate before calling

// validate embedding credentials before boot-triggered rebuild
if (!process.env.EMBEDDING_API_KEY && process.env.AGENTMEMORY_EMBEDDING_PROVIDER !== "local") {
  console.warn("Embedding provider key unset — index rebuild will fail");
}

Type guard

null

Try / catch

rebuildIndex(kv)
  .then((n) => console.log(`index rebuilt: ${n}`))
  .catch((err) => console.warn(`[agentmemory] Failed to rebuild search index:`, err));

Prevention

Set and verify embedding provider API keys before first boot
Expect a rebuild whenever BM25 starts empty (e.g. after a load failure) and fix the load error too
Handle provider rate limits for large corpora; rebuild fills lazily from traffic as fallback
Keep memory records schema-clean across upgrades; prune legacy malformed entries

When it happens

Trigger: Boot with `bm25Index.size === 0` triggering `rebuildIndex`, and the rebuild rejects — embedding provider unreachable/rate-limited (429/5xx), invalid API key, malformed observations in KV, KV list failure.

Common situations: Missing or wrong embedding API credentials; offline machine; rate limits on large corpora; legacy records from older versions that fail validation during indexing; the empty-index state itself often results from error 73 (load failed).

Related errors

AI-assisted analysis of rohitg00/agentmemory@e04ba88819 (2026-08-30). Data as JSON: /api/errors/3711cf0dd81f0400. Report an issue: GitHub.