{"record":{"id":"231b70641ebc12f1","repo":"xai-org/x-algorithm","slug":"embeddingcache-missing-emb-dim-metadata","errorCode":null,"errorMessage":"EmbeddingCache: missing emb_dim metadata","messagePattern":"EmbeddingCache: missing emb_dim metadata","errorType":"error_code","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"phoenix/crates/serving/xai-recsys-mm-server/src/embedding_cache.rs","lineNumber":284,"sourceCode":"        let data_mdb = lmdb_path.join(\"data.mdb\");\n        let map_size = std::fs::metadata(&data_mdb)\n            .map(|m| m.len() as usize)\n            .unwrap_or(0)\n            .max(1024 * 1024);\n\n        let env = unsafe {\n            heed::EnvOpenOptions::new()\n                .map_size(map_size)\n                .max_readers(1024)\n                .open(lmdb_path)?\n        };\n        let rtxn = env.read_txn()?;\n        let db = env.open_database(&rtxn, None)?.ok_or_else(|| {\n            anyhow::anyhow!(\"EmbeddingCache: no database in {}\", lmdb_path.display())\n        })?;\n        let emb_dim = db\n            .get(&rtxn, &LMDB_META_KEY_EMB_DIM)?\n            .ok_or_else(|| anyhow::anyhow!(\"EmbeddingCache: missing emb_dim metadata\"))?\n            as usize;\n        rtxn.commit()?;\n\n        let file = OpenOptions::new().read(true).write(true).open(data_path)?;\n        let mmap = unsafe { MmapMut::map_mut(&file)? };\n        let slot_size = SLOT_HEADER_SIZE + emb_dim * std::mem::size_of::<f16>();\n        let capacity = (mmap.len() - HEADER_SIZE) / slot_size;\n\n        log::info!(\n            \"EmbeddingCache: opened lmdb={} data={} (capacity={}, emb_dim={})\",\n            lmdb_path.display(),\n            data_path.display(),\n            capacity,\n            emb_dim,\n        );\n\n        Ok(Self {\n            env,","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/crates/serving/xai-recsys-mm-server/src/embedding_cache.rs#L266-L302","documentation":"Raised by EmbeddingCache::open when the LMDB database exists but the meta key LMDB_META_KEY_EMB_DIM is absent. The embedding dimension is required to compute slot size (SLOT_HEADER_SIZE + emb_dim * sizeof(f16)) for the mmap region, so the cache cannot be laid out.","triggerScenarios":"Calling open() on an LMDB database that was populated by an older version of create() that didn't write emb_dim metadata, or by an external writer; partially-written/corrupt metadata.","commonSituations":"Version skew between the tool that wrote the cache and the one reading it; interrupted create() that wrote records but not metadata; manual LMDB population without the meta key.","solutions":["Regenerate the cache with the current version of create() so emb_dim metadata is written","If you control the writer, ensure it stores LMDB_META_KEY_EMB_DIM before committing","Pass emb_dim explicitly or add a fallback (infer from data file size / slot size) if regeneration is too costly","Add a cache-format version key alongside emb_dim to detect format mismatches early"],"exampleFix":"// before\nlet emb_dim = db.get(&rtxn, &LMDB_META_KEY_EMB_DIM)?\n    .ok_or_else(|| anyhow::anyhow!(\"EmbeddingCache: missing emb_dim metadata\"))? as usize;\n\n// after\nlet emb_dim = match db.get(&rtxn, &LMDB_META_KEY_EMB_DIM)? {\n    Some(v) => v as usize,\n    None => {\n        let slot = data_file_size / num_slots; // fallback inference\n        if slot > SLOT_HEADER_SIZE { (slot - SLOT_HEADER_SIZE) / std::mem::size_of::<f16>() }\n        else { anyhow::bail!(\"EmbeddingCache: missing emb_dim metadata and cannot infer\") }\n    }\n};","handlingStrategy":"validation","validationCode":"// before open(): check meta key presence with a quick read_txn; if absent, regenerate cache","typeGuard":null,"tryCatchPattern":"null","preventionTips":["Write metadata atomically with data in create()","Bump a cache-format version key on schema changes","Refuse to open caches from older writers"],"tags":["lmdb","embedding-cache","metadata","version-skew","rust"],"backgroundTag":"missing-metadata-key","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}