What causes "litellm embedding response has {len(data)} items for {n} inputs" in cocoindex-io/cocoindex?

Reduce the batch size of texts sent per embedding request and retry.

cocoindex-io/cocoindex · error · RuntimeError

litellm embedding response has

Error message

litellm embedding response has {len(data)} items for {n} inputs

What it means

Raised by `_aligned_embeddings` when a litellm embedding response contains a different number of items than the number of input texts sent. The library aligns each returned embedding to its input text and cannot do so safely when counts differ, so it fails loudly rather than silently misaligning embeddings.

Solutions

Reduce the batch size of texts sent per embedding request and retry.
Inspect the raw litellm response (log it) to see whether the provider actually returned fewer items.
Check the provider/model route for known response-shape issues, or switch provider/model.
Upgrade litellm and cocoindex if a recent version change altered the response format.

Example fix

// before
embeddings = embed_op.embed(texts)  # texts has 500 items, provider truncates
// after
for chunk in _chunks(texts, 64):
    embeddings.extend(embed_op.embed(chunk))

Defensive patterns

Strategy: retry

Validate before calling

if isinstance(resp, dict) and len(resp.get("data", [])) != len(texts):
    raise ValueError(f"provider returned {len(resp.get('data', []))} items for {len(texts)} inputs")

Try / catch

try:
    embs = embed_op.embed(texts)
except RuntimeError as e:
    if 'embedding response has' in str(e):
        embs = [embed_op.embed([t])[0] for t in texts]  # fall back to per-item calls
    else:
        raise

Prevention

Keep embedding batches modest (e.g. <=128) to avoid provider truncation.
Log raw responses once per new provider/model to verify shape before production use.
Pin litellm and provider SDK versions; re-verify after upgrades.

When it happens

Trigger: Calling a litellm-based embedding op where the provider (or litellm proxy) returns fewer or more embedding items in `data` than the number of input strings, e.g. dropped inputs, provider truncation, or batching bugs.

Common situations: Provider-side truncation on very large batches; a litellm proxy aggregating/rewriting responses; using a model/route that returns a partial response; API version changes altering the response shape.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/a6b711198ff97371. Report an issue: GitHub.

Appendix: source

Thrown at python/cocoindex/ops/litellm.py:181

            multiplier=2.0,
            max_delay=_EMBEDDING_RETRY_MAX_BACKOFF_SECONDS,
        ),
        bound_attempt=True,
        operation_name=operation_name,
    )


def _aligned_embeddings(data: list[_Any], n: int) -> list[_NDArray[_np.float32]]:
    """Map embedding response items back to the ``n`` inputs they embed.

    Items carrying an ``index`` are placed by it; if no item carries one
    (missing or ``None``), the response is taken positionally. Mixing the two,
    or an index set that is not a permutation of ``0..n-1``, raises so a
    misordered response fails loudly instead of silently misaligning
    embeddings with their texts.
    """
    if len(data) != n:
        raise RuntimeError(
            f"litellm embedding response has {len(data)} items for {n} inputs"
        )
    out: list[_NDArray[_np.float32] | None] = [None] * n
    indexed = n > 0 and data[0].get("index") is not None
    for pos, item in enumerate(data):
        index = item.get("index")
        if (index is not None) != indexed:
            raise RuntimeError(
                "litellm embedding response mixes items with and without `index`"
            )
        if not indexed:
            index = pos
        elif type(index) is not int or not 0 <= index < n or out[index] is not None:
            raise RuntimeError(
                "litellm embedding response indices are not a permutation of "
                f"0..{n - 1}: got {[item.get('index') for item in data]}"
            )
        out[index] = _np.array(item["embedding"], dtype=_np.float32)

View on GitHub (pinned to e84aa99b32)