RyanCodrai/turbovec · error · ValueError

documents have empty embeddings (dim 0); check the embedder

Error message

documents have empty embeddings (dim 0); check the embedder that produced them

What it means

Raised in _commit_batch when the batch is 2D but with second dimension 0 — i.e. every document has a zero-length embedding, shape (N, 0). This passes the ndim guard but would otherwise die deep in the index kernel with an opaque buffer-length error; the store names the real cause: the embedder produced empty vectors.

Source

Thrown at turbovec-python/python/turbovec/haystack.py:344

        precedes any mutation, and if the index add fails the
        pre-inserted map entries are unwound (restoring the previous
        mapping of any overwritten id), so a failure leaves the store
        exactly as it was (issue #89). The FAIL path calls this with
        single-document batches to get per-document commit semantics.
        Callers hold the writer lock.
        """
        vectors = np.asarray(
            [doc.embedding for doc in to_write], dtype=np.float32
        )
        if vectors.ndim != 2:
            raise ValueError(
                f"expected 2D embedding batch, got {vectors.ndim}D"
            )
        # A batch of empty per-document embeddings has shape (N, 0) — 2D,
        # so it passes the ndim guard, then dies deep in the index kernel
        # with an opaque buffer-length error. Name the real cause instead.
        if vectors.shape[1] == 0:
            raise ValueError(
                "documents have empty embeddings (dim 0); check the "
                "embedder that produced them"
            )
        # IdMapIndex.add_with_ids handles both eager (dim must match) and
        # lazy (locks dim on first call) cases. Surface its mismatch
        # panic as a clean ValueError for parity with previous behaviour.
        existing_dim = self._index.dim
        if existing_dim is not None and vectors.shape[1] != existing_dim:
            raise ValueError(
                f"embedding dim {vectors.shape[1]} does not match store dim {existing_dim}"
            )
        if not vectors.flags["C_CONTIGUOUS"]:
            vectors = np.ascontiguousarray(vectors)
        # Cosine mode: L2-normalize so the kernel's raw score is true
        # cosine similarity. Pure numpy on the just-built batch (no
        # embedder call — Haystack documents arrive pre-embedded), so
        # doing it alongside the rest of the batch prep under the
        # caller's writer lock adds no blocking work. Zero rows pass

View on GitHub (pinned to ccab9f325e)

Solutions

1. Check the embedder component that produced the documents — empty vectors usually mean an empty input text path or a broken model.
2. Validate embedding length at embed time before handing documents to the store.
3. Catch the ValueError in pipeline code to fail the indexing step with a clear embedder-related message.

When it happens

Trigger: Thrown at turbovec-python/python/turbovec/haystack.py:344 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/e3da8a81d7decbf2. Report an issue: GitHub.