unslothai/unsloth · error · RuntimeError

embedder returned ragged vectors: shape {arr.shape}

Error message

embedder returned ragged vectors: shape {arr.shape}

What it means

Raised after batching when the collected embedding rows cannot be stacked into a 2-D (N, dim) array — i.e. vectors have inconsistent lengths (ragged). np.asarray of rows with differing dims yields an object/array with ndim != 2, which the backend rejects because downstream cosine similarity and the vector store assume fixed-width rows. It indicates the server returned vectors of varying dimensionality, typically from a model swap or a malformed response.

Source

Thrown at studio/backend/core/rag/embed_llama_server.py:760

        rows: list[list[float]] = []
        batch = max(1, config.EMBED_BATCH)
        for start in range(0, n, batch):
            chunk = list(texts[start : start + batch])
            data = self._post(
                "/v1/embeddings",
                {"input": chunk, "model": "embedding", "encoding_format": "float"},
            )
            items = data.get("data", [])
            if len(items) != len(chunk):
                raise RuntimeError(
                    f"embedder returned {len(items)} vectors for {len(chunk)} inputs"
                )
            # OpenAI spec lets the server reorder; sort back by index.
            items = sorted(items, key = lambda d: d.get("index", 0))
            rows.extend(d["embedding"] for d in items)
        arr = np.asarray(rows, dtype = np.float32)
        if arr.ndim != 2:
            raise RuntimeError(f"embedder returned ragged vectors: shape {arr.shape}")
        if normalize:
            norms = np.linalg.norm(arr, axis = 1, keepdims = True)
            norms[norms == 0] = 1.0
            arr = arr / norms
        return arr

    def dim(self, *, model_name = None) -> int:
        """Embedding width, probed via a 1-text encode and cached per model
        (_resolve_model_path clears it when the effective repo changes).
        Unlocked: concurrent probes are benign, and locking would deadlock when
        the probe's encode respawns onto a changed model (see __init__)."""
        self._ensure_ready()
        cached = self._dim
        if cached is not None:
            return cached
        vec = self.encode(["x"], normalize = False)
        width = int(vec.shape[1])
        self._dim = width

View on GitHub (pinned to 203007d190)

Solutions

1. Log arr.shape and the set of len(row) values to identify which batch produced odd-length vectors.
2. Ensure the embedding model (GGUF) cannot change mid-run — restart ingestion workers after changing EMBED_MODEL_PATH.
3. Verify the GGUF is a genuine embedding model with a fixed output dimension (probe with a single-text encode and check dim()).
4. Re-download the GGUF if corrupted; verify its checksum.

Example fix

# before
arr = np.asarray(rows, dtype=np.float32)

# after (diagnostic guard before stacking)
widths = {len(r) for r in rows}
if len(widths) > 1:
    raise RuntimeError(f"ragged embedding widths {sorted(widths)}; server model changed mid-call?")
arr = np.asarray(rows, dtype=np.float32)

widths = {len(r) for r in rows}
if len(widths) != 1:
    raise ValueError(f"server returned mixed embedding widths {sorted(widths)}")
# only then stack
arr = np.asarray(rows, dtype=np.float32)

try:
    arr = np.asarray(rows, dtype=np.float32)
    if arr.ndim != 2:
        raise RuntimeError(f"ragged vectors: shape {arr.shape}")
except RuntimeError:
    # dimension drifted mid-call: restart worker so all batches use one model
    request_backend_reload()

When it happens

Trigger: Multiple encode() batches in one call straddling a server restart onto a different GGUF with a different embedding dimension; a server bug returning truncated floats or wrong-length vectors for some inputs; mixing responses from different models when the server was externally restarted mid-call.

Common situations: Changing EMBED_MODEL_PATH while a worker is mid-ingest so early batches use the old dimension and later batches the new; concurrent processes each spawning servers with different models; a quantized GGUF whose output layer is inconsistent.

Related errors

• embedder returned {len(items)} vectors for {len(chunk)} inpu
• Unknown RAG_EMBED_BACKEND={config.EMBED_BACKEND!r}; expected
• Could not verify {model!r} as an embedding model on Hugging
• load error: {p.get('error')}
• model load did not reach ready within {timeout_s}s

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/fb5a1832ee5b6575. Report an issue: GitHub.