cocoindex-io/cocoindex · error · RuntimeError

litellm embedding response mixes items with and without…

Error message

litellm embedding response mixes items with and without `index`

What it means

Raised when a litellm embedding response contains a mix of items with and without an `index` field. The alignment logic decides globally whether the response is index-aligned or positional (based on the first item); a mixture is ambiguous, so the library raises instead of guessing.

Solutions

Log/inspect the raw response to identify which provider or route produces the inconsistent items.
If using a litellm proxy/router, pin the batch to a single provider/deployment so all items share one response format.
Normalize the response before it reaches cocoindex: add `index` to every item (its position) or strip it from all items.
Upgrade litellm if the provider's response format changed recently.

Example fix

// before
data = response["data"]  # some items lack "index"
// after
data = [{**item, "index": i} for i, item in enumerate(response["data"])]

Defensive patterns

Strategy: validation

Validate before calling

data = resp["data"]
indexed_flags = {item.get("index") is not None for item in data}
if len(indexed_flags) > 1:
    raise ValueError("response mixes indexed and positional items")

Try / catch

try:
    embs = embed_op.embed(texts)
except RuntimeError as e:
    if 'mixes items with and without' in str(e):
        resp = normalize_indices(raw_response)
        embs = embed_op.embed(texts)
    else:
        raise

Prevention

Route a batch to a single provider/deployment through the litellm router.
Normalize proxy responses to one format (all-indexed or all-positional) before feeding the pipeline.
Add a smoke test per provider that asserts a uniform response shape.

When it happens

Trigger: The provider returns `data` items where some entries include `"index": n` and others omit it or set it to None — e.g. a proxy merging responses from different backends or a partially-migrated API format.

Common situations: litellm proxy fanning out a batch to multiple providers; custom gateways rewriting response items; provider SDK upgrades changing whether index is emitted; hand-rolled mock servers used in tests.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/f37cf66a0699bab4. Report an issue: GitHub.

Appendix: source

Thrown at python/cocoindex/ops/litellm.py:189

def _aligned_embeddings(data: list[_Any], n: int) -> list[_NDArray[_np.float32]]:
    """Map embedding response items back to the ``n`` inputs they embed.

    Items carrying an ``index`` are placed by it; if no item carries one
    (missing or ``None``), the response is taken positionally. Mixing the two,
    or an index set that is not a permutation of ``0..n-1``, raises so a
    misordered response fails loudly instead of silently misaligning
    embeddings with their texts.
    """
    if len(data) != n:
        raise RuntimeError(
            f"litellm embedding response has {len(data)} items for {n} inputs"
        )
    out: list[_NDArray[_np.float32] | None] = [None] * n
    indexed = n > 0 and data[0].get("index") is not None
    for pos, item in enumerate(data):
        index = item.get("index")
        if (index is not None) != indexed:
            raise RuntimeError(
                "litellm embedding response mixes items with and without `index`"
            )
        if not indexed:
            index = pos
        elif type(index) is not int or not 0 <= index < n or out[index] is not None:
            raise RuntimeError(
                "litellm embedding response indices are not a permutation of "
                f"0..{n - 1}: got {[item.get('index') for item in data]}"
            )
        out[index] = _np.array(item["embedding"], dtype=_np.float32)
    return _cast(list[_NDArray[_np.float32]], out)


class LiteLLMEmbedder(_schema.VectorSchemaProvider):
    """Wrapper for LiteLLM embedding models that implements VectorSchemaProvider.

    This class provides an async interface to LiteLLM's embedding API
    and automatically provides vector schema information for CocoIndex connectors.

View on GitHub (pinned to e84aa99b32)