unslothai/unsloth · error · RuntimeError

embedder returned {len(items)} vectors for {len(chunk)} inpu

Error message

embedder returned {len(items)} vectors for {len(chunk)} inputs

What it means

Raised during encode() when the /v1/embeddings response contains a different number of embedding objects than texts sent in the batch. The backend requires a strict 1:1 correspondence before it sorts by index and stacks the vectors, so a mismatched count is treated as a protocol violation rather than silently producing misaligned rows. It guards against server bugs or truncated responses corrupting the vector store.

Source

Thrown at studio/backend/core/rag/embed_llama_server.py:752

        model_name = None,
        normalize = True,
    ):
        """Embed texts -> (N, dim) float32. ``model_name`` is ignored (the GGUF is
        fixed by config). Normalizes in Python to match the ST backend."""
        n = len(texts)
        if n == 0:
            return np.zeros((0, self.dim()), dtype = np.float32)
        rows: list[list[float]] = []
        batch = max(1, config.EMBED_BATCH)
        for start in range(0, n, batch):
            chunk = list(texts[start : start + batch])
            data = self._post(
                "/v1/embeddings",
                {"input": chunk, "model": "embedding", "encoding_format": "float"},
            )
            items = data.get("data", [])
            if len(items) != len(chunk):
                raise RuntimeError(
                    f"embedder returned {len(items)} vectors for {len(chunk)} inputs"
                )
            # OpenAI spec lets the server reorder; sort back by index.
            items = sorted(items, key = lambda d: d.get("index", 0))
            rows.extend(d["embedding"] for d in items)
        arr = np.asarray(rows, dtype = np.float32)
        if arr.ndim != 2:
            raise RuntimeError(f"embedder returned ragged vectors: shape {arr.shape}")
        if normalize:
            norms = np.linalg.norm(arr, axis = 1, keepdims = True)
            norms[norms == 0] = 1.0
            arr = arr / norms
        return arr

    def dim(self, *, model_name = None) -> int:
        """Embedding width, probed via a 1-text encode and cached per model
        (_resolve_model_path clears it when the effective repo changes).
        Unlocked: concurrent probes are benign, and locking would deadlock when

View on GitHub (pinned to 203007d190)

Solutions

Log len(chunk) and len(items) at the failure to see whether the server is dropping, capping, or adding items.
Filter or pad empty/whitespace-only strings out of texts before calling encode().
Lower EMBED_BATCH below any server-side batch limit.
Pin/upgrade llama.cpp to a version with a conformant /v1/embeddings implementation.
If using a custom server, verify its response against the OpenAI embeddings schema (data[i].index present, one item per input).

Example fix

# before
texts = [chunk.page_content for chunk in chunks]  # may contain ""

# after
texts = [t if t.strip() else " " for t in (c.page_content for c in chunks)]  # no empty inputs

Defensive patterns

Strategy: validation

Validate before calling

def sanitize_texts(texts: list[str]) -> list[str]:
    # no empty/whitespace inputs; servers may drop them and break the 1:1 count
    return [t if t.strip() else " " for t in texts]

Try / catch

try:
    arr = backend.encode(chunk_texts)
except RuntimeError as e:
    if "vectors for" in str(e) and "inputs" in str(e):
        # retry one-by-one to isolate the offending text, or shrink batch
        arr = np.stack([backend.encode([t])[0] for t in chunk_texts])
    else:
        raise

Prevention

Never send empty strings to /v1/embeddings; substitute a single space.
Keep EMBED_BATCH below the server's documented per-request item limit.
Pin a llama.cpp version whose /v1/embeddings you have verified returns exactly N items.

When it happens

Trigger: Calling encode() with a batch where llama-server drops or duplicates items (server bug, streaming truncation, non-OpenAI-compliant server behind the URL); a server that errors on empty-string inputs and omits them from the response; EMBED_BATCH exceeding a server-side limit that silently caps response items.

Common situations: Pointing the backend at a proxy or alternative OpenAI-compatible server that implements /v1/embeddings loosely; llama.cpp server versions with batch embedding bugs; documents that reduce to empty strings after preprocessing.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/87c19e6928103c89. Report an issue: GitHub.