{"record":{"id":"290fc91a6672d5c6","repo":"rohitg00/agentmemory","slug":"agentmemory-persisted-vector-index-has-mismatc","errorCode":null,"errorMessage":"[agentmemory] Persisted vector index has ${mismatches.length} of ${loaded.vector.size} vectors with the wrong dimension. Active provider (${embeddingProvider?.name}) declares ${activeDim}; dimensions seen on disk: ${distinct}. AGENTMEMORY_DROP_STALE_INDEX=true is set — discarding the persisted vectors. Live observations will rebuild the index over time.","messagePattern":"\\[agentmemory\\] Persisted vector index has (.+?) of (.+?) vectors with the wrong dimension\\. Active provider \\((.+?)\\) declares (.+?); dimensions seen on disk: (.+?)\\. AGENTMEMORY_DROP_STALE_INDEX=true is set — discarding the persisted vectors\\. Live observations will rebuild the index over time\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/index.ts","lineNumber":437,"sourceCode":"    // 0 on cross-dim pairs, so affected observations stop matching\n    // anything and recall degrades without an error. Walk every stored\n    // vector instead of trusting the first; refuse to load if anything\n    // is off.\n    const activeDim = embeddingProvider?.dimensions ?? 0;\n    const { mismatches, seenDimensions } =\n      activeDim > 0\n        ? loaded.vector.validateDimensions(activeDim)\n        : { mismatches: [], seenDimensions: new Set<number>() };\n\n    if (mismatches.length > 0) {\n      const sample = mismatches\n        .slice(0, 5)\n        .map((m) => `${m.obsId} (dim=${m.dim})`)\n        .join(\", \");\n      const distinct = Array.from(seenDimensions).sort((a, b) => a - b).join(\", \");\n      const dropStale = isDropStaleIndexEnabled();\n      if (dropStale) {\n        console.warn(\n          `[agentmemory] Persisted vector index has ${mismatches.length} of ` +\n            `${loaded.vector.size} vectors with the wrong dimension. Active ` +\n            `provider (${embeddingProvider?.name}) declares ${activeDim}; ` +\n            `dimensions seen on disk: ${distinct}. ` +\n            `AGENTMEMORY_DROP_STALE_INDEX=true is set — discarding the persisted ` +\n            `vectors. Live observations will rebuild the index over time.`,\n        );\n      } else {\n        throw new Error(\n          `[agentmemory] Refusing to start: persisted vector index has ` +\n            `${mismatches.length} of ${loaded.vector.size} vectors with the ` +\n            `wrong dimension. Active provider (${embeddingProvider?.name}) ` +\n            `declares ${activeDim}; dimensions seen on disk: ${distinct}. ` +\n            `First mismatched obsIds: ${sample}. Loading would silently corrupt ` +\n            `search (cross-dimension cosine returns 0). Choose one:\\n` +\n            `  - Re-embed the existing index against the new provider, then start.\\n` +\n            `  - Set AGENTMEMORY_DROP_STALE_INDEX=true to discard the persisted ` +\n            `vectors and rebuild from live observations.\\n` +","sourceCodeStart":419,"sourceCodeEnd":455,"githubUrl":"https://github.com/rohitg00/agentmemory/blob/e04ba88819c365c9acf9d6661ea802143e728bd6/src/index.ts#L419-L455","documentation":"At startup, main() loads the persisted vector index and compares each vector's dimension to the active embedding provider's declared dimension. When mismatches exist and AGENTMEMORY_DROP_STALE_INDEX=true is set, it warns loudly and discards the persisted vectors; the index is rebuilt from live observations over time. This prevents dimension-mismatched vectors from corrupting similarity search after an embedding model change.","triggerScenarios":"main() startup: loaded.vector.size > 0 with mismatches.length > 0, isDropStaleIndexEnabled() true — i.e. the on-disk index was built with a different embedding dimension than the currently configured provider declares.","commonSituations":"Switching embedding providers/models (e.g. 1536-dim OpenAI to 768-dim local) without clearing data; restoring an old data directory; config change of AGENTMEMORY_EMBEDDING_* variables while stale index remains.","solutions":["No action needed if the drop is intended — the index rebuilds from live observations.","If you did NOT intend to change dimensions, restore the previous embedding provider config to keep existing vectors.","Back up ./data before switching embedding providers so you can roll back.","Re-embed historical observations if you need old data searchable under the new provider."],"exampleFix":"// before: provider switched, data silently dropped\nAGENTMEMORY_DROP_STALE_INDEX=true\n// after: pin the provider matching the persisted vectors, or back up first\ncp -r ./data ./data.bak  # then switch providers intentionally","handlingStrategy":"fallback","validationCode":"// before switching providers, compare declared dimension to persisted index\nconst declaredDim = provider.dimensions;\nif (persistedDim && persistedDim !== declaredDim) {\n  console.warn(`dimension change ${persistedDim} -> ${declaredDim}; index will be dropped/rebuilt`);\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Back up ./data before changing embedding providers.","Re-embed historical observations after an intentional provider switch.","Keep AGENTMEMORY_EMBEDDING_* config stable across restarts.","Read startup warnings — they reveal silent index drops."],"tags":["vector-index","embeddings","dimension-mismatch","data-migration"],"backgroundTag":"embedding-dimension-mismatch","analyzedSha":"e04ba88819c365c9acf9d6661ea802143e728bd6","analyzedAt":"2026-08-30T01:07:40.754Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}