{"record":{"id":"ea02df7dd7632062","repo":"RyanCodrai/turbovec","slug":"node-embedding-dim-vectors-shape-1-does-not-mat","errorCode":null,"errorMessage":"node embedding dim {vectors.shape[1]} does not match index dim {existing_dim}","messagePattern":"node embedding dim (.+?) does not match index dim (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"turbovec-python/python/turbovec/llama_index.py","lineNumber":401,"sourceCode":"        # PARENT / CHILD), excluded_*_metadata_keys, template fields,\n        # start/end_char_idx and mimetype on retrieval. The narrow\n        # `{text, metadata, ref_doc_id}` schema we used to keep lost\n        # all of those silently.\n        payloads = [_payload_for(node) for node in nodes]\n\n        # Cosine mode: L2-normalize outside the lock (pure computation)\n        # so the engine's raw inner product is true cosine similarity.\n        # Zero rows pass through unchanged.\n        if self._similarity == COSINE:\n            vectors = l2_normalize_rows(vectors)\n\n        with self._write_lock:\n            # IdMapIndex.add_with_ids handles eager (dim must match) and lazy\n            # (locks dim on first add) — pre-check the eager case so we\n            # surface a clean ValueError rather than a Rust panic.\n            existing_dim = self._index.dim\n            if existing_dim is not None and vectors.shape[1] != existing_dim:\n                raise ValueError(\n                    f\"node embedding dim {vectors.shape[1]} does not match index dim {existing_dim}\"\n                )\n            if not vectors.flags[\"C_CONTIGUOUS\"]:\n                vectors = np.ascontiguousarray(vectors)\n\n            handles = np.array([self._issue_handle() for _ in nodes], dtype=np.uint64)\n\n            # Capture the previous state of any upserted node_id BEFORE the\n            # maps are overwritten, so a failed index add can restore it and\n            # the old vectors can be dropped once the add succeeds.\n            old = [\n                (nid, self._node_id_to_u64[nid], self._nodes[nid])\n                for nid in node_ids\n                if nid in self._node_id_to_u64\n            ]\n\n            # Maps BEFORE the index add: a concurrent query can only learn\n            # a handle from the index, so an entry that is resolvable but","sourceCodeStart":383,"sourceCodeEnd":419,"githubUrl":"https://github.com/RyanCodrai/turbovec/blob/ccab9f325e6ce2a270a87daf01ae4e443bcf2d49/turbovec-python/python/turbovec/llama_index.py#L383-L419","documentation":"The store's underlying index already has a fixed vector dimension and the incoming node embeddings have a different width. For an eager (already-populated) index, adding mismatched-dim vectors would cause a Rust panic, so the Python layer pre-checks and raises a clean ValueError.","triggerScenarios":"Calling add() with embeddings of dimension D2 when the index was created/first-populated with dimension D1 — typically after switching embed models against an existing persisted index.","commonSituations":"Changing from one embed model (e.g. 384-dim MiniLM) to another (e.g. 1536-dim OpenAI) while reusing a persisted turbovec index; index dim locked by a previous first insert.","solutions":["Rebuild the index from scratch with the new embed model instead of appending to the old one","Ensure all documents are embedded with the same model used when the index dim was locked","Delete the persisted index files and re-ingest the corpus"],"exampleFix":"// before\nstore = TurboQuantVectorStore(persist_path=\"old_index.tqv\")  # 384-dim\nstore.add(new_1536_dim_nodes)\n// after\nstore = TurboQuantVectorStore(persist_path=None)  # fresh index for new dim\nstore.add(new_1536_dim_nodes)","handlingStrategy":"validation","validationCode":"dim = store._index.dim\nif dim is not None and len(nodes[0].get_embedding()) != dim:\n    raise ValueError(f\"embedding dim {len(nodes[0].get_embedding())} != index dim {dim}\")","typeGuard":"def dims_match(nodes, index_dim: int | None) -> bool:\n    return index_dim is None or all(len(n.get_embedding()) == index_dim for n in nodes)","tryCatchPattern":"try:\n    store.add(nodes)\nexcept ValueError as e:\n    if \"does not match index dim\" in str(e):\n        raise RuntimeError(\"switched embed models? rebuild the index from scratch\") from e\n    raise","preventionTips":["Never reuse a persisted index after changing embed models","Record the embed model name/id alongside the index","Version index files by embedding dim"],"tags":["python","dimension-mismatch","vector-store","embedding"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"ccab9f325e6ce2a270a87daf01ae4e443bcf2d49","analyzedAt":"2026-09-06T08:39:18.516Z","contentChangedAt":"2026-09-06T08:39:18.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}