{"record":{"id":"3711cf0dd81f0400","repo":"rohitg00/agentmemory","slug":"agentmemory-failed-to-rebuild-search-index","errorCode":null,"errorMessage":"[agentmemory] Failed to rebuild search index:","messagePattern":"\\[agentmemory\\] Failed to rebuild search index:","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/index.ts","lineNumber":486,"sourceCode":"\n  if (needsRebuild) {\n    // Fire-and-forget. rebuildIndex iterates every observation across\n    // every session and AWAITS an embedding-provider call per record.\n    // On a large corpus + rate-limited embedding endpoint that can\n    // take HOURS; awaiting it here blocks every subsequent boot step\n    // (including startViewerServer below, leaving the viewer port\n    // unbound for the duration). The index lazily fills in over time\n    // and search degrades gracefully — partial coverage > no viewer\n    // for hours. Errors still surface via the inner .catch.\n    void rebuildIndex(kv)\n      .then((indexCount) => {\n        if (indexCount > 0) {\n          bootLog(`Search index rebuilt: ${indexCount} entries`);\n          indexPersistence.scheduleSave();\n        }\n      })\n      .catch((err) => {\n        console.warn(`[agentmemory] Failed to rebuild search index:`, err);\n      });\n  } else {\n    // Backfill memories into BM25 for users upgrading from <0.9.5: prior\n    // versions of mem::remember never indexed memories, so the persisted\n    // BM25 covers observations only and `memory_smart_search` returns\n    // empty for everything saved via memory_save (#257). Walk KV.memories\n    // and add the ones missing from the restored index. Idempotent on\n    // re-runs because SearchIndex.has() short-circuits already-indexed\n    // ids.\n    try {\n      const memories = await kv.list<import(\"./types.js\").Memory>(KV.memories);\n      let backfilled = 0;\n      for (const memory of memories) {\n        if (memory.isLatest === false) continue;\n        if (!memory.title || !memory.content) continue;\n        if (bm25Index.has(memory.id)) continue;\n        bm25Index.add({\n          id: memory.id,","sourceCodeStart":468,"sourceCodeEnd":504,"githubUrl":"https://github.com/rohitg00/agentmemory/blob/e04ba88819c365c9acf9d6661ea802143e728bd6/src/index.ts#L468-L504","documentation":"Warning logged when the fire-and-forget background rebuild (`rebuildIndex(kv)`) rejects. This runs at boot when the BM25 index is empty, iterating every observation and calling the embedding provider per record; any error inside (provider/network failure, corrupted KV records) is caught here so boot is never blocked or crashed. Search stays partially indexed until a successful rebuild.","triggerScenarios":"Boot with `bm25Index.size === 0` triggering `rebuildIndex`, and the rebuild rejects — embedding provider unreachable/rate-limited (429/5xx), invalid API key, malformed observations in KV, KV list failure.","commonSituations":"Missing or wrong embedding API credentials; offline machine; rate limits on large corpora; legacy records from older versions that fail validation during indexing; the empty-index state itself often results from error 73 (load failed).","solutions":["Inspect the logged `err` — provider HTTP status vs data error","Fix embedding credentials/env (e.g. provider API key, AGENTMEMORY_URL) and restart to retrigger rebuild (it runs whenever BM25 is empty)","For rate limits, reduce corpus or use a provider with higher quota; the index also fills lazily from live traffic","If caused by malformed legacy records, prune/repair those KV entries or upgrade to a version that skips them","Set AGENTMEMORY_DROP_STALE_INDEX=true if stale persisted vectors are involved, then restart"],"exampleFix":"// before\n[agentmemory] Failed to rebuild search index: Error: 401 Unauthorized from embedding provider\n// after\nexport EMBEDDING_API_KEY=sk-...\n$ agentmemory start\nSearch index rebuilt: 1204 entries","handlingStrategy":"try-catch","validationCode":"// validate embedding credentials before boot-triggered rebuild\nif (!process.env.EMBEDDING_API_KEY && process.env.AGENTMEMORY_EMBEDDING_PROVIDER !== \"local\") {\n  console.warn(\"Embedding provider key unset — index rebuild will fail\");\n}","typeGuard":"null","tryCatchPattern":"rebuildIndex(kv)\n  .then((n) => console.log(`index rebuilt: ${n}`))\n  .catch((err) => console.warn(`[agentmemory] Failed to rebuild search index:`, err));","preventionTips":["Set and verify embedding provider API keys before first boot","Expect a rebuild whenever BM25 starts empty (e.g. after a load failure) and fix the load error too","Handle provider rate limits for large corpora; rebuild fills lazily from traffic as fallback","Keep memory records schema-clean across upgrades; prune legacy malformed entries"],"tags":["startup","search","bm25","embedding-provider","background-rebuild"],"backgroundTag":"index-rebuild-failed","analyzedSha":"e04ba88819c365c9acf9d6661ea802143e728bd6","analyzedAt":"2026-08-30T01:07:40.754Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}