RyanCodrai/turbovec · error · ValueError

failed to embed {len(missing)} document(s): {ids}

Error message

failed to embed {len(missing)} document(s): {ids}

What it means

insert() verifies every agno document received a non-empty embedding (the embedder was expected to fill doc.embedding). If any documents still have embedding None or empty, a ValueError lists how many and which ids failed, because the quantized index cannot ingest un-embedded documents.

Source

Thrown at turbovec-python/python/turbovec/agno.py:501

            for doc in documents:
                meta = dict(doc.meta_data) if doc.meta_data else {}
                meta.update(filters)
                doc.meta_data = meta

        self._embed_missing(documents)

        # Raise on any document that still lacks an embedding rather than
        # silently dropping — silent drops mask data-pipeline bugs.
        # None/len check instead of truthiness: `not <ndarray>` raises the
        # numpy truth-value-ambiguous ValueError (issue #135).
        missing = [
            doc
            for doc in documents
            if doc.embedding is None or len(doc.embedding) == 0
        ]
        if missing:
            ids = [doc.id or "<no id>" for doc in missing]
            raise ValueError(
                f"failed to embed {len(missing)} document(s): {ids}"
            )

        # Batch the entire `documents` list into a single add_with_ids call.
        # Per-document inserts would invalidate the SIMD-blocked cache
        # between every doc.
        vectors = np.asarray([doc.embedding for doc in documents], dtype=np.float32)
        if vectors.ndim != 2:
            raise ValueError(
                f"expected 2D embedding batch, got {vectors.ndim}D"
            )
        if vectors.shape[1] != self.dimensions:
            raise ValueError(
                f"embedding dim {vectors.shape[1]} does not match "
                f"index dim {self.dimensions}"
            )
        if not vectors.flags["C_CONTIGUOUS"]:
            vectors = np.ascontiguousarray(vectors)

View on GitHub (pinned to ccab9f325e)

Solutions

1. Embed documents before insert: docs = embedder.get_embedding_and_use(docs) (or the async equivalent).
2. Log/inspect the listed failing ids — check their content for empty strings or inputs the embedder rejects.
3. Add retry/error handling around the embedder call so transient API failures don't produce None embeddings.
4. Filter out or skip documents that fail embedding instead of passing them through to insert().

Example fix

// before
vec_db.insert(documents)  # documents have embedding=None
// after
embedded = embedder.get_embedding_and_use(documents)
vec_db.insert(embedded)

unembedded = [d for d in documents if not d.embedding]
if unembedded:
    documents = embedder.get_embedding_and_use(documents)

def is_embedded(doc) -> bool:
    return doc.embedding is not None and len(doc.embedding) > 0

try:
    db.insert(content_hash, documents)
except ValueError as e:
    logger.error("embedding failed for some docs: %s", e)
    documents = [d for d in documents if is_embedded(d)]  # retry without failures

When it happens

Trigger: Calling insert() (or upsert(), which delegates to insert) with documents whose embedding is None/empty — usually when documents were constructed with embedding=None relying on the DB to embed, or the embedder silently returned empty vectors for some inputs.

Common situations: Embedding API rate limits or errors that yield no embedding for some docs; passing pre-baked Document objects without embeddings assuming auto-embed; embedder/model misconfiguration producing empty output for empty or oversized content.

Understand the failure class

Background: "must not be empty", "cannot be empty" — required-field validation errors across open-source libraries — this error's family across 41 libraries.

Related errors

• `embedder` is required; turbovec needs the embedder's `dimen
• nodes have empty embeddings (dim 0); check the embed model t
• duplicate id in batch: {k!r}
• {prefix} {version}; this turbovec accepts versions {list(com
• persisted store is corrupt: {len(extraneous)} {what} id(s) p

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/d9c2d5788c28c23c. Report an issue: GitHub.