RyanCodrai/turbovec · error · ValueError

expected 2D embedding batch, got {vectors.ndim}D

Error message

expected 2D embedding batch, got {vectors.ndim}D

What it means

Raised in _commit_batch after stacking the batch's embeddings with np.asarray: the result is not a 2D (N, dim) matrix. Since write_documents already rejects missing embeddings, this points at malformed embedding payloads — ragged lists, scalars, or non-uniform shapes — that cannot form a uniform matrix.

Source

Thrown at turbovec-python/python/turbovec/haystack.py:337

        add the vectors to the index (dropping ``to_remove``'s old
        handles last, index first).

        Maps BEFORE the index add: a concurrent retrieval can only learn
        a handle from the index, so an entry that is resolvable but not
        yet searchable is invisible to readers (issue #161). Still
        all-or-nothing with respect to the given batch: validation
        precedes any mutation, and if the index add fails the
        pre-inserted map entries are unwound (restoring the previous
        mapping of any overwritten id), so a failure leaves the store
        exactly as it was (issue #89). The FAIL path calls this with
        single-document batches to get per-document commit semantics.
        Callers hold the writer lock.
        """
        vectors = np.asarray(
            [doc.embedding for doc in to_write], dtype=np.float32
        )
        if vectors.ndim != 2:
            raise ValueError(
                f"expected 2D embedding batch, got {vectors.ndim}D"
            )
        # A batch of empty per-document embeddings has shape (N, 0) — 2D,
        # so it passes the ndim guard, then dies deep in the index kernel
        # with an opaque buffer-length error. Name the real cause instead.
        if vectors.shape[1] == 0:
            raise ValueError(
                "documents have empty embeddings (dim 0); check the "
                "embedder that produced them"
            )
        # IdMapIndex.add_with_ids handles both eager (dim must match) and
        # lazy (locks dim on first call) cases. Surface its mismatch
        # panic as a clean ValueError for parity with previous behaviour.
        existing_dim = self._index.dim
        if existing_dim is not None and vectors.shape[1] != existing_dim:
            raise ValueError(
                f"embedding dim {vectors.shape[1]} does not match store dim {existing_dim}"
            )

View on GitHub (pinned to ccab9f325e)

Solutions

1. Inspect the offending documents' embeddings for ragged lengths or nested arrays and re-embed with a healthy embedder.
2. Ensure every document's embedding is a flat 1D sequence of floats of equal length.
3. Catch the ValueError to abort the write before any index mutation (validation precedes mutation, so state is unchanged).

When it happens

Trigger: Thrown at turbovec-python/python/turbovec/haystack.py:337 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/72a6e095751918ee. Report an issue: GitHub.