{"record":{"id":"98e58572c5a673a8","repo":"rohitg00/agentmemory","slug":"agentmemory-refusing-to-start-persisted-vector","errorCode":null,"errorMessage":"[agentmemory] Refusing to start: persisted vector index has ${mismatches.length} of ${loaded.vector.size} vectors with the wrong dimension. Active provider (${embeddingProvider?.name}) declares ${activeDim}; dimensions seen on disk: ${distinct}. First mismatched obsIds: ${sample}. Loading would silently corrupt search (cross-dimension cosine returns 0). Choose one:\n  - Re-embed the existing index against the new provider, then start.\n  - Set AGENTMEMORY_DROP_STALE_INDEX=true to discard the persisted vectors and start fresh.","messagePattern":"\\[agentmemory\\] Refusing to start: persisted vector index has (.+?) of (.+?) vectors with the wrong dimension\\. Active provider \\((.+?)\\) declares (.+?); dimensions seen on disk: (.+?)\\. First mismatched obsIds: (.+?)\\. Loading would silently corrupt search \\(cross-dimension cosine returns 0\\)\\. Choose one:\n  - Re-embed the existing index against the new provider, then start\\.\n  - Set AGENTMEMORY_DROP_STALE_INDEX=true to discard the persisted vectors and start fresh\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"src/index.ts","lineNumber":446,"sourceCode":"\n    if (mismatches.length > 0) {\n      const sample = mismatches\n        .slice(0, 5)\n        .map((m) => `${m.obsId} (dim=${m.dim})`)\n        .join(\", \");\n      const distinct = Array.from(seenDimensions).sort((a, b) => a - b).join(\", \");\n      const dropStale = isDropStaleIndexEnabled();\n      if (dropStale) {\n        console.warn(\n          `[agentmemory] Persisted vector index has ${mismatches.length} of ` +\n            `${loaded.vector.size} vectors with the wrong dimension. Active ` +\n            `provider (${embeddingProvider?.name}) declares ${activeDim}; ` +\n            `dimensions seen on disk: ${distinct}. ` +\n            `AGENTMEMORY_DROP_STALE_INDEX=true is set — discarding the persisted ` +\n            `vectors. Live observations will rebuild the index over time.`,\n        );\n      } else {\n        throw new Error(\n          `[agentmemory] Refusing to start: persisted vector index has ` +\n            `${mismatches.length} of ${loaded.vector.size} vectors with the ` +\n            `wrong dimension. Active provider (${embeddingProvider?.name}) ` +\n            `declares ${activeDim}; dimensions seen on disk: ${distinct}. ` +\n            `First mismatched obsIds: ${sample}. Loading would silently corrupt ` +\n            `search (cross-dimension cosine returns 0). Choose one:\\n` +\n            `  - Re-embed the existing index against the new provider, then start.\\n` +\n            `  - Set AGENTMEMORY_DROP_STALE_INDEX=true to discard the persisted ` +\n            `vectors and rebuild from live observations.\\n` +\n            `  - Switch the embedding provider back to the one that wrote the index.`,\n        );\n      }\n    } else {\n      vectorIndex.restoreFrom(loaded.vector);\n      bootLog(\n        `Loaded persisted vector index (${vectorIndex.size} vectors)`,\n      );\n    }","sourceCodeStart":428,"sourceCodeEnd":464,"githubUrl":"https://github.com/rohitg00/agentmemory/blob/e04ba88819c365c9acf9d6661ea802143e728bd6/src/index.ts#L428-L464","documentation":"At startup, agentmemory loads the persisted vector index and verifies each stored vector's dimension against the active embedding provider. If mismatches exist and AGENTMEMORY_DROP_STALE_INDEX is not set, main() throws to prevent cross-dimension cosine comparisons that would silently corrupt search results. The error names the offending observation ids and offers re-embedding or dropping the index.","triggerScenarios":"Switching embedding providers/models (e.g. from a 384-dim model to a 1536-dim one) and restarting agentmemory against the existing ./data/state_store.db vector index, without AGENTMEMORY_DROP_STALE_INDEX=true.","commonSituations":"Upgrading agentmemory or changing EMBEDDING_PROVIDER/EMBEDDING_MODEL config on a deployment with existing observations; moving a data directory from one project config to another.","solutions":["Re-embed the persisted index against the new provider, then restart.","Set AGENTMEMORY_DROP_STALE_INDEX=true to discard stale vectors and start fresh (observations text is kept; the index rebuilds over time).","Revert the embedding provider/model config to the one that produced the persisted vectors."],"exampleFix":"// before\nEMBEDDING_PROVIDER=openai  # switched model, old 384-dim vectors on disk\n// after\n# either re-embed, or explicitly accept data loss:\nAGENTMEMORY_DROP_STALE_INDEX=true EMBEDDING_PROVIDER=openai npm start","handlingStrategy":"validation","validationCode":"// before starting, check the persisted index against the provider\nconst providerDim = embeddingProvider.dimensions;\nconst onDisk = loadPersistedDims(); // from data/state_store.db\nconst stale = onDisk.filter((d) => d !== providerDim);\nif (stale.length > 0 && process.env.AGENTMEMORY_DROP_STALE_INDEX !== \"true\") {\n  process.env.AGENTMEMORY_DROP_STALE_INDEX = \"true\"; // or run re-embed job first\n}","typeGuard":null,"tryCatchPattern":"try {\n  await startAgentMemory();\n} catch (e) {\n  if (String(e.message).includes(\"wrong dimension\")) {\n    // do NOT auto-drop in prod; alert and require a re-embed or explicit opt-in\n    process.env.AGENTMEMORY_DROP_STALE_INDEX = \"true\";\n    return startAgentMemory();\n  }\n  throw e;\n}","preventionTips":["Pin EMBEDDING_PROVIDER/EMBEDDING_MODEL in config and treat changes as a migration event.","Record the embedding model + dimension in deployment metadata and diff on deploy.","When changing models, always run a re-embed job — reserve AGENTMEMORY_DROP_STALE_INDEX for dev environments."],"tags":["embeddings","startup","data-migration"],"backgroundTag":"vector-dimension-mismatch","analyzedSha":"e04ba88819c365c9acf9d6661ea802143e728bd6","analyzedAt":"2026-08-30T01:07:40.754Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}