RyanCodrai/turbovec · error · ValueError

embedding dimension {vectors.shape[1]} does not match index

Error message

embedding dimension {vectors.shape[1]} does not match index dim {existing_dim}

What it means

Raised in _store_texts_and_vectors under the write lock when the batch's embedding width differs from the index's committed dim (self._index.dim). Pre-checked so the eager-dimension mismatch surfaces as a clean ValueError rather than a Rust panic; the embedder used for these texts differs from the one the index was built with.

Source

Thrown at turbovec-python/python/turbovec/langchain.py:407

            ids = [ids[i] for i in keep]
            texts_list = [texts_list[i] for i in keep]
            metadatas = [metadatas[i] for i in keep]
            vectors = vectors[keep]

        # Cosine mode: L2-normalize outside the lock (pure computation,
        # like the embedding step) so the engine's raw inner product is
        # true cosine similarity. Zero rows pass through unchanged.
        if self._similarity == COSINE:
            vectors = l2_normalize_rows(vectors)

        with self._write_lock:
            # Validate before mutating any existing data. IdMapIndex.add_with_ids
            # handles both eager (dim must match) and lazy (locks dim on first
            # call) cases. Pre-check the eager case so we surface a clean
            # ValueError rather than a Rust panic.
            existing_dim = self._index.dim
            if existing_dim is not None and vectors.shape[1] != existing_dim:
                raise ValueError(
                    f"embedding dimension {vectors.shape[1]} does not match index dim {existing_dim}"
                )
            if not vectors.flags["C_CONTIGUOUS"]:
                vectors = np.ascontiguousarray(vectors)

            handles = np.array(
                [self._issue_handle() for _ in texts_list], dtype=np.uint64
            )

            # Capture the previous state of any upserted id BEFORE the maps
            # are overwritten, so a failed index add can restore it and the
            # old vectors can be dropped once the add succeeds.
            old = [
                (i, self._str_to_u64[i], self._docs[i])
                for i in ids
                if i in self._str_to_u64
            ]

View on GitHub (pinned to ccab9f325e)

Solutions

1. Re-embed texts with the same embedder used to create the index.
2. Create a fresh TurboQuantVectorStore if the embedding model intentionally changed.
3. Catch the ValueError to detect embedder/store drift before any vector is written.

When it happens

Trigger: Thrown at turbovec-python/python/turbovec/langchain.py:407 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/21d6b0ab0cee57f6. Report an issue: GitHub.