RyanCodrai/turbovec · error · ValueError

embedding dim {vectors.shape[1]} does not match store dim {e

Error message

embedding dim {vectors.shape[1]} does not match store dim {existing_dim}

What it means

Raised in _commit_batch when the batch's embedding width differs from the store's committed dimension (self._index.dim, set at first write or from load). It converts what would be a Rust-side mismatch panic into a clean ValueError: documents were embedded with a different model/size than the store was built with.

Source

Thrown at turbovec-python/python/turbovec/haystack.py:353

        )
        if vectors.ndim != 2:
            raise ValueError(
                f"expected 2D embedding batch, got {vectors.ndim}D"
            )
        # A batch of empty per-document embeddings has shape (N, 0) — 2D,
        # so it passes the ndim guard, then dies deep in the index kernel
        # with an opaque buffer-length error. Name the real cause instead.
        if vectors.shape[1] == 0:
            raise ValueError(
                "documents have empty embeddings (dim 0); check the "
                "embedder that produced them"
            )
        # IdMapIndex.add_with_ids handles both eager (dim must match) and
        # lazy (locks dim on first call) cases. Surface its mismatch
        # panic as a clean ValueError for parity with previous behaviour.
        existing_dim = self._index.dim
        if existing_dim is not None and vectors.shape[1] != existing_dim:
            raise ValueError(
                f"embedding dim {vectors.shape[1]} does not match store dim {existing_dim}"
            )
        if not vectors.flags["C_CONTIGUOUS"]:
            vectors = np.ascontiguousarray(vectors)
        # Cosine mode: L2-normalize so the kernel's raw score is true
        # cosine similarity. Pure numpy on the just-built batch (no
        # embedder call — Haystack documents arrive pre-embedded), so
        # doing it alongside the rest of the batch prep under the
        # caller's writer lock adds no blocking work. Zero rows pass
        # through unchanged.
        if self._vectors_normalized:
            vectors = l2_normalize_rows(vectors)

        handles = np.array(
            [self._issue_handle() for _ in to_write], dtype=np.uint64
        )

        # Capture the previous handle of every overwritten id BEFORE the

View on GitHub (pinned to ccab9f325e)

Solutions

1. Re-embed documents with the same embedder used for the store's existing vectors.
2. Create a fresh store if the embedding model intentionally changed.
3. Catch the ValueError in ingestion pipelines to detect embedder/store configuration drift early.

When it happens

Trigger: Thrown at turbovec-python/python/turbovec/haystack.py:353 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/978467813eda3da9. Report an issue: GitHub.