BerriAI/litellm · error · OCIError

oci_serving_mode must be 'ON_DEMAND' or 'DEDICATED'.

Error message

oci_serving_mode must be 'ON_DEMAND' or 'DEDICATED'.

What it means

OCI generative-ai embedding models can be served two ways: ON_DEMAND (shared, identified by modelId) or DEDICATED (a dedicated cluster, identified by endpointId). The adapter reads `optional_params['oci_serving_mode']` (default 'ON_DEMAND'), uppercases it, and rejects anything outside {'ON_DEMAND', 'DEDICATED'} with this 400 error.

Source

Thrown at litellm/llms/oci/embed/transformation.py:222

                            "Convert token lists to strings before calling embedding()."
                        ),
                    )
                texts.append(item if isinstance(item, str) else str(item))
        else:
            texts = [str(input)]

        if len(texts) > OCI_EMBED_BATCH_LIMIT:
            raise OCIError(
                status_code=400,
                message=(
                    f"OCI embedText accepts at most {OCI_EMBED_BATCH_LIMIT} inputs per request "
                    f"(got {len(texts)}). Batch your requests."
                ),
            )

        serving_mode_type: Final = optional_params.get("oci_serving_mode", "ON_DEMAND").upper()
        if serving_mode_type not in {"ON_DEMAND", "DEDICATED"}:
            raise OCIError(
                status_code=400,
                message="oci_serving_mode must be 'ON_DEMAND' or 'DEDICATED'.",
            )

        if serving_mode_type == "DEDICATED":
            endpoint_id: Final = optional_params.get("oci_endpoint_id", model)
            serving_mode = OCIServingMode(servingType="DEDICATED", endpointId=endpoint_id)
        else:
            serving_mode = OCIServingMode(servingType="ON_DEMAND", modelId=model)

        # Map input_type from OpenAI convention to OCI/Cohere convention
        input_type = optional_params.get("input_type")
        if input_type:
            input_type = _INPUT_TYPE_MAP.get(input_type.lower(), input_type.upper())

        request: Final = OCIEmbedRequest(
            compartmentId=compartment_id,
            servingMode=serving_mode,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Set the parameter to exactly 'ON_DEMAND' or 'DEDICATED' (case-insensitive): `litellm.embedding(..., oci_serving_mode='DEDICATED')`.
2. If using DEDICATED, also pass `oci_endpoint_id` so the request targets your endpoint (it defaults to the model string otherwise).
3. Remove the parameter entirely if you want the default on-demand serving.

Example fix

# before
resp = litellm.embedding(model="oci/cohere.embed-v3", input=["hi"], oci_serving_mode="DEDICATED_CLUSTER")

# after
resp = litellm.embedding(model="oci/cohere.embed-v3", input=["hi"], oci_serving_mode="DEDICATED", oci_endpoint_id="ocid1.generativeaiendpoint.oc1...")

SERVING_MODES = {"ON_DEMAND", "DEDICATED"}
mode = (oci_serving_mode or "ON_DEMAND").upper()
if mode not in SERVING_MODES:
    raise ValueError(f"oci_serving_mode must be one of {SERVING_MODES}, got {oci_serving_mode!r}")

When it happens

Trigger: Passing `oci_serving_mode='dedicated-cluster'`, `'batch'`, or a typo like `'ONDEMAND'` (no underscore) as an extra kwarg to litellm.embedding with an oci/ model. Any value that does not equal ON_DEMAND or DEDICATED after .upper() triggers it.

Common situations: Copy-pasting serving mode names from the OCI console or SDK docs that use different casing/labels, or mistakenly passing the OCI SDK enum object instead of its string value. Users switching from dedicated endpoints to on-demand and editing the param by hand.

Related errors

• Private key file not found: {file_path}
• Private key file is empty: {file_path}
• Invalid OCI region {region!r}: must match ^[a-z][a-z0-9-]{{0
• Missing required OCI credentials: oci_user, oci_fingerprint,
• Missing required parameters: oci_user, oci_fingerprint, oci_

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/a2fc581dfda2c64f. Report an issue: GitHub.