BerriAI/litellm · error · OCIError

{response.text}

Error message

{response.text}

What it means

Raised when the synchronous streaming request to OCI Generative AI completes but the response status code is not 200. The full response body (OCI's error JSON) is attached to an OCIError with the upstream status code, telling you why the service refused the streaming chat request.

Source

Thrown at litellm/llms/oci/chat/transformation.py:661

        signed_json_body: bytes | None = None,
    ) -> "OCIStreamWrapper":
        if client is None or isinstance(client, AsyncHTTPHandler):
            client = _get_httpx_client(params={})

        try:
            response: Final = client.post(
                api_base,
                headers=headers,
                data=(signed_json_body if signed_json_body is not None else json.dumps(data)),
                stream=True,
                logging_obj=logging_obj,
                timeout=STREAMING_TIMEOUT,
            )
        except httpx.HTTPStatusError as e:
            raise OCIError(status_code=e.response.status_code, message=e.response.text)

        if response.status_code != 200:
            raise OCIError(status_code=response.status_code, message=response.text)

        return OCIStreamWrapper(
            completion_stream=_iter_sse_events(response.iter_text()),
            model=model,
            custom_llm_provider=custom_llm_provider,
            logging_obj=logging_obj,
        )

    @track_llm_api_timing()
    async def get_async_custom_stream_wrapper(
        self,
        model: str,
        custom_llm_provider: str,
        logging_obj: LiteLLMLoggingObj,
        api_base: str,
        headers: dict,
        data: dict,
        messages: list,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read e.status_code and e.message (the OCI error JSON) to identify the exact refusal reason.
2. For 400: strip unsupported parameters (e.g. provider-specific params OCI rejects) or fix the model OCID.
3. For 401/403: re-check OCI credentials (user OCID, fingerprint, tenancy, key) and that the key is RSA.
4. For 429/5xx: retry with exponential backoff and jitter, or reduce request concurrency.

Example fix

# before
stream = litellm.completion(model="oci/cohere.command-r-plus", messages=m, stream=True)

# after
from litellm.llms.oci.common_utils import OCIError
try:
    stream = litellm.completion(model="oci/cohere.command-r-plus", messages=m, stream=True)
except OCIError as e:
    if e.status_code and 500 <= e.status_code < 600 and attempt < 3:
        continue  # retry loop
    raise

import re
model = "ocid1.generativeai.model.oc1....."
region = os.environ.get("OCI_REGION", "")
assert re.match(r"^[a-z][a-z0-9-]{0,30}[a-z0-9]$", region.strip()), "bad OCI_REGION"

def is_retryable_oci_status(code: int | None) -> bool:
    return code == 429 or (code is not None and 500 <= code < 600)

from litellm.llms.oci.common_utils import OCIError
try:
    stream = litellm.completion(model="oci/...", messages=m, stream=True)
except OCIError as e:
    if is_retryable_oci_status(e.status_code) and attempt < MAX:
        time.sleep(2 ** attempt + random.random())
        continue
    raise

When it happens

Trigger: litellm.completion(..., custom_llm_provider='oci' or model='oci/...', stream=True) where OCI returns any non-200: 400 invalid request payload, 401 signing failure, 404 unknown model OCID/action path, 429 throttling, 5xx service error.

Common situations: Model OCID copied from a different region or compartment; request payload includes parameters OCI does not accept for that model; tenancy not subscribed to Generative AI; stale api_base override that no longer routes to the inference endpoint; sustained load hitting the service limit.

Related errors

• {e.response.text}
• _body (masked response body)
• str(response.read())
• str(await response.aread())
• Chunk cannot be parsed as CohereStreamChunk: {e}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/fa31f4a8c69df21a. Report an issue: GitHub.