BerriAI/litellm · error · OCIError

Chunk cannot be parsed as CohereStreamChunk

Error message

Chunk cannot be parsed as CohereStreamChunk: {e}

What it means

While streaming an OCI Cohere completion, each SSE event is parsed into the pydantic model CohereStreamChunk. If a chunk's JSON does not validate (TypeError/ValidationError), LiteLLM raises OCIError (500) and aborts the stream mid-iteration. Like 1682 this indicates the streamed chunk shape diverges from what this LiteLLM version expects — a server-side/packaging mismatch, not malformed caller input.

Solutions

1. Upgrade LiteLLM to the latest release — streaming chunk parsers are patched as OCI formats evolve
2. Reproduce with stream=False first: if the non-streaming call works, the issue is specific to the SSE chunk shape; capture chunks (litellm.client or httpx event hook) and compare against the embedded pydantic error
3. Retry the request — transient corrupt SSE frames (network truncation) also land here; a fresh connection usually succeeds
4. Try a different OCI region or cohere model version to sidestep a model-specific stream quirk
5. If persistent, file a litellm issue with the offending sanitized chunk; meanwhile pin the last known-good litellm version

Example fix

# before
stream = litellm.completion(model="oci/cohere.command-r-16k", messages=msgs, stream=True)
for chunk in stream:  # OCIError: Chunk cannot be parsed as CohereStreamChunk
    ...

# after: guard the iteration and fall back to non-streaming
from litellm.exceptions import OCIError
try:
    for chunk in litellm.completion(model="oci/cohere.command-r-16k", messages=msgs, stream=True):
        print(chunk.choices[0].delta.content or "", end="")
except OCIError:
    resp = litellm.completion(model="oci/cohere.command-r-16k", messages=msgs)  # no stream
    print(resp.choices[0].message.content)

from litellm.exceptions import OCIError

try:
    text = ""
    for chunk in litellm.completion(model=MODEL, messages=msgs, stream=True):
        text += chunk.choices[0].delta.content or ""
except OCIError as e:
    if "CohereStreamChunk" not in str(e):
        raise
    resp = litellm.completion(model=MODEL, messages=msgs)  # degrade to non-streaming
    text = resp.choices[0].message.content

When it happens

Trigger: litellm.completion(..., stream=True) with an oci/cohere.* model where an SSE event fails CohereStreamChunk validation — e.g. OCI emits a new keep-alive/event type, a field changes type between intermediate and terminal chunks, or an error event arrives in the SSE channel.

Common situations: OCI rolls out stream-format changes per region or model version; dedicated-AI-cluster firmware serving an older/newer SSE contract than public OCI; old litellm version against updated endpoints; sporadic proxy/LB-injected non-JSON lines into the SSE channel.

Related errors

• Response cannot be casted to CohereChatResult
• Error parsing chunk: , Received chunk
• Failed to decode JSON from chunk
• Chat provider: Empty parsed_chunk
• Chat provider: Invalid chunk type

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/c5da6c590ccc0aff. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/oci/chat/cohere.py:284

    ``prior_tool_calls_emitted`` lets the caller signal whether tool calls
    were already emitted in earlier chunks of the same stream. When set, the
    terminal consolidation chunk's tool calls are suppressed (they would
    duplicate prior deltas); otherwise they are passed through so a stream
    that delivers tool calls only on the terminal chunk doesn't silently
    drop them.

    ``prior_text_emitted`` plays the analogous role for the ``text`` field:
    when set, the terminal consolidation chunk's ``text`` is suppressed
    (it would re-emit the full assembled response on top of prior deltas);
    when unset (e.g. a degenerate stream that delivers the entire response
    in a single SSE event carrying both ``chatHistory`` and ``finishReason``),
    the text is passed through so the response content isn't silently lost.
    """
    try:
        typed_chunk: Final = CohereStreamChunk(**dict_chunk)
    except (TypeError, ValidationError) as e:
        raise OCIError(
            status_code=500,
            message=f"Chunk cannot be parsed as CohereStreamChunk: {e}",
        )

    if typed_chunk.index is None:
        typed_chunk.index = 0

    # OCI Cohere's terminal SSE event re-sends the full assembled response in
    # `text` alongside a populated `chatHistory` and a non-null `finishReason`.
    # Emitting that text would concatenate the whole response onto the
    # already-streamed deltas. We require both signals to be present so that a
    # future API change which adds `chatHistory` to intermediate chunks (or a
    # rare early-populated case) doesn't silently drop legitimate token deltas.
    is_terminal_consolidation: Final = typed_chunk.chatHistory is not None and typed_chunk.finishReason is not None
    # On non-terminal text-free chunks (e.g. tool-call-only or keep-alive
    # chunks) emit ``content=None`` rather than ``content=""`` so downstream
    # stream-mergers that distinguish "no text in this delta" from "an
    # explicitly empty text delta" behave correctly.

View on GitHub (pinned to 77b7c6c40c)