BerriAI/litellm · error · OCIError

OCI embedText accepts at most {OCI_EMBED_BATCH_LIMIT} inputs

Error message

OCI embedText accepts at most {OCI_EMBED_BATCH_LIMIT} inputs per request (got {len(texts)}). Batch your requests.

What it means

The OCI embedText endpoint enforces a hard per-request batch limit, which LiteLLM's OCI adapter defines as OCI_EMBED_BATCH_LIMIT = 96 (litellm/llms/oci/embed/transformation.py:56). After flattening `input` into a list of strings, if there are more than 96 texts the adapter raises this 400 error client-side instead of sending a doomed request.

Source

Thrown at litellm/llms/oci/embed/transformation.py:212

        if isinstance(input, str):
            texts = [input]
        elif isinstance(input, list):
            texts = []
            for item in input:
                if isinstance(item, list):
                    raise OCIError(
                        status_code=400,
                        message=(
                            "OCI embedText does not support token-array inputs. "
                            "Convert token lists to strings before calling embedding()."
                        ),
                    )
                texts.append(item if isinstance(item, str) else str(item))
        else:
            texts = [str(input)]

        if len(texts) > OCI_EMBED_BATCH_LIMIT:
            raise OCIError(
                status_code=400,
                message=(
                    f"OCI embedText accepts at most {OCI_EMBED_BATCH_LIMIT} inputs per request "
                    f"(got {len(texts)}). Batch your requests."
                ),
            )

        serving_mode_type: Final = optional_params.get("oci_serving_mode", "ON_DEMAND").upper()
        if serving_mode_type not in {"ON_DEMAND", "DEDICATED"}:
            raise OCIError(
                status_code=400,
                message="oci_serving_mode must be 'ON_DEMAND' or 'DEDICATED'.",
            )

        if serving_mode_type == "DEDICATED":
            endpoint_id: Final = optional_params.get("oci_endpoint_id", model)
            serving_mode = OCIServingMode(servingType="DEDICATED", endpointId=endpoint_id)
        else:

View on GitHub (pinned to 6c2dcb801b)

Solutions

Split your input into chunks of at most 96 texts and loop: `for batch in [input[i:i+96] for i in range(0, len(input), 96)]: ...`.
Use litellm's batching utilities or a task queue for very large corpora instead of one giant call.
Double-check you are not accidentally passing a list of characters (e.g. `list(text)`) which inflates the count.

Example fix

# before
resp = litellm.embedding(model="oci/generative-ai-cohere-embed-v3", input=docs)  # len(docs) = 500

# after
embeddings = []
for i in range(0, len(docs), 96):
    batch = docs[i:i+96]
    resp = litellm.embedding(model="oci/generative-ai-cohere-embed-v3", input=batch)
    embeddings.extend(d["embedding"] for d in resp.data)

Defensive patterns

Strategy: validation

Validate before calling

OCI_BATCH = 96
batches = [input[i:i+OCI_BATCH] for i in range(0, len(input), OCI_BATCH)]
assert all(len(b) <= OCI_BATCH for b in batches)

Prevention

Chunk inputs to <=96 texts before calling oci/ embeddings
Wrap the loop in a helper so every call site is batch-safe
Log batch sizes during ingestion to catch accidental oversized calls

When it happens

Trigger: Calling litellm.embedding with an OCI embedding model and an input list longer than 96 strings, e.g. embedding a 500-document corpus in one call: `litellm.embedding(model='oci/generative-ai-cohere-embed-v3', input=docs)` where len(docs) > 96.

Common situations: Bulk-embedding document stores, RAG ingestion pipelines, or migrating from providers with larger batch limits (OpenAI allows 2048+) without re-chunking. A single string input also becomes a 1-element list, so this only fires for genuinely large batches.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/51fd0ff09a7e805b. Report an issue: GitHub.