cocoindex-io/cocoindex · error · RuntimeError

litellm embedding response indices are not a permutation of…

Error message

litellm embedding response indices are not a permutation of 0..{n - 1}: got {[item.get('index') for item in data]}

What it means

Raised when the `index` values in a litellm embedding response are not a valid permutation of 0..n-1 — i.e. an index is not an int, is out of range, or duplicates another index. Positional alignment is impossible in that case, so the library fails loudly rather than placing embeddings at wrong positions.

Solutions

Log the full list of returned indices (included in the error message) to identify the off-by-one or duplication pattern.
If indices are 1-based from the provider, normalize them (subtract 1) or make the request positional by stripping `index` before passing to cocoindex.
Stop sharding the same batch across multiple backend calls, or fix the proxy so indices are unique and 0-based.
Retry with smaller batch sizes to rule out provider-side batch handling bugs.

Example fix

// before
# provider returns 1-based indices
resp = embed(texts)  # raises: indices not a permutation of 0..n-1
// after
for item in resp["data"]:
    item["index"] = int(item["index"]) - 1  # normalize to 0-based

Defensive patterns

Strategy: validation

Validate before calling

idx = [item.get("index") for item in resp["data"]]
if sorted(i for i in idx if isinstance(i, int)) != list(range(len(resp["data"]))):
    raise ValueError(f"indices not a permutation: {idx}")

Type guard

def is_valid_permutation(indices, n):
    return len(indices) == n and all(type(i) is int and 0 <= i < n for i in indices) and len(set(indices)) == n

Try / catch

try:
    embs = embed_op.embed(texts)
except RuntimeError as e:
    if 'not a permutation' in str(e):
        resp = reindex_positionally(raw_response)  # drop provider indices
        embs = embed_op.embed(texts)
    else:
        raise

Prevention

Check provider index conventions (0-based vs 1-based) once per integration and normalize.
Avoid splitting one logical batch across multiple backend calls that renumber indices.
Validate indices with a smoke test before wiring a new model route into production.

When it happens

Trigger: Provider returns duplicate `index` values, indices >= number of inputs, negative indices, or non-integer indices (e.g. floats or strings) in the embedding response `data` items.

Common situations: Proxies or routers that merge/shard batch requests and renumber incorrectly; providers emitting 1-based indices instead of 0-based; index emitted as string due to JSON formatting; sending n inputs but the provider reports indices relative to a different batch.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/5e0e0887c024859c. Report an issue: GitHub.

Appendix: source

Thrown at python/cocoindex/ops/litellm.py:195

    misordered response fails loudly instead of silently misaligning
    embeddings with their texts.
    """
    if len(data) != n:
        raise RuntimeError(
            f"litellm embedding response has {len(data)} items for {n} inputs"
        )
    out: list[_NDArray[_np.float32] | None] = [None] * n
    indexed = n > 0 and data[0].get("index") is not None
    for pos, item in enumerate(data):
        index = item.get("index")
        if (index is not None) != indexed:
            raise RuntimeError(
                "litellm embedding response mixes items with and without `index`"
            )
        if not indexed:
            index = pos
        elif type(index) is not int or not 0 <= index < n or out[index] is not None:
            raise RuntimeError(
                "litellm embedding response indices are not a permutation of "
                f"0..{n - 1}: got {[item.get('index') for item in data]}"
            )
        out[index] = _np.array(item["embedding"], dtype=_np.float32)
    return _cast(list[_NDArray[_np.float32]], out)


class LiteLLMEmbedder(_schema.VectorSchemaProvider):
    """Wrapper for LiteLLM embedding models that implements VectorSchemaProvider.

    This class provides an async interface to LiteLLM's embedding API
    and automatically provides vector schema information for CocoIndex connectors.

    Args:
        model: LiteLLM model name (e.g., ``"text-embedding-ada-002"``,
            ``"vertex_ai/textembedding-gecko"``).
        **kwargs: Additional keyword arguments passed through to every
            ``litellm.aembedding`` call (e.g., ``api_key``, ``api_base``,

View on GitHub (pinned to e84aa99b32)