BerriAI/litellm · error · OCIError

Response cannot be casted to CohereChatResult

Error message

Response cannot be casted to CohereChatResult: {e}

What it means

After a successful HTTP call to OCI's Cohere chat endpoint, LiteLLM validates the JSON body against the pydantic model CohereChatResult. If the payload has missing/wrongly-typed fields (TypeError or pydantic ValidationError), it raises OCIError with the upstream HTTP status code. This almost always means the OCI response shape does not match what this LiteLLM version expects — upstream API drift, not a bad request you sent.

Solutions

1. Upgrade LiteLLM: pip install -U litellm — parser models track OCI schema changes and fixes land regularly
2. Capture the raw response to inspect the mismatch: temporarily wrap with litellm client kwargs or use a proxy to log the upstream body, then compare fields against the error text (it embeds the pydantic error)
3. Try another OCI region or the same model on the OCID-serving endpoint to rule out a region-specific rollout
4. Retry once — transient OCI gateway error pages (HTML/JSON error in 200) also trigger this
5. If it persists, report with the sanitized response body to the litellm GitHub issue tracker; downgrade to the last working litellm version as a stopgap

Example fix

# before
resp = litellm.completion(model="oci/cohere.command-r-plus-08-2024", messages=msgs)
# OCIError: Response cannot be casted to CohereChatResult: ...

# after: pin/upgrade a litellm release matching current OCI schema, and guard the call
import litellm
from litellm.exceptions import OCIError
try:
    resp = litellm.completion(model="oci/cohere.command-r-plus-08-2024", messages=msgs)
except OCIError:
    resp = litellm.completion(model="oci/generic.model", messages=msgs)  # GENERIC handler path

from litellm.exceptions import OCIError

try:
    resp = litellm.completion(model=MODEL, messages=msgs)
except OCIError as e:
    if "cannot be casted to CohereChatResult" in str(e):
        resp = litellm.completion(model=FALLBACK_MODEL, messages=msgs)  # e.g. oci/generic.* or another vendor
    else:
        raise

When it happens

Trigger: Non-streaming completion via model="oci/cohere.<variant>" (e.g. cohere.command-r-plus-08-2024 served through OCI Generative AI) where the returned JSON fails CohereChatResult validation — e.g.OCI changed field names/casing, returned an error-shaped 200 body, or a model returned an undocumented structure.

Common situations: OCI updates their Cohere response contract (region rollout differences); pinning an old litellm against a new OCI API, or new litellm against an OCI Dedicated AI cluster running legacy firmware; model not enabled in the compartment returning a structured error inside a 200; isStreaming=false hitting a handler path only exercised for specific cohere versions.

Related errors

• Chunk cannot be parsed as CohereStreamChunk
• api_base is required
• Cannot create domain model from None record
• CohereException
• Content item of type `text` must have a string `text` field

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/e2c7c9f81e0337c1. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/oci/chat/cohere.py:203

                description=function_def.get("description", ""),
                parameterDefinitions=parameter_definitions,
            )
        )

    return cohere_tools


def handle_cohere_response(
    json_response: dict,
    model: str,
    model_response: ModelResponse,
    raw_response: httpx.Response,
) -> ModelResponse:
    """Parse a non-streaming Cohere OCI response into a LiteLLM ModelResponse."""
    try:
        cohere_response: Final = CohereChatResult(**json_response)
    except (TypeError, ValidationError) as e:
        raise OCIError(
            message=f"Response cannot be casted to CohereChatResult: {e}",
            status_code=raw_response.status_code,
        )

    model_response.model = model
    model_response.created = int(datetime.datetime.now().timestamp())

    response_text: Final = cohere_response.chatResponse.text
    finish_reason: Final = _normalize_oci_finish_reason(cohere_response.chatResponse.finishReason)

    tool_calls: list[dict[str, Any]] | None = None
    if cohere_response.chatResponse.toolCalls:
        tool_calls = [
            {
                "id": _synthesize_oci_tool_call_id(i, tc.name, json.dumps(tc.parameters, sort_keys=True)),
                "type": "function",
                "function": {
                    "name": tc.name,

View on GitHub (pinned to 77b7c6c40c)