BerriAI/litellm · error · OCIError

Chunk cannot be parsed as CohereStreamChunk: {e}

Error message

Chunk cannot be parsed as CohereStreamChunk: {e}

What it means

During streaming, each SSE chunk from OCI Cohere is parsed into the `CohereStreamChunk` Pydantic model. A chunk whose fields do not fit raises OCIError(500) 'Chunk cannot be parsed as CohereStreamChunk: {details}'. Unlike the non-streaming variant this always uses status 500, and it can fire mid-stream after valid chunks were already consumed — callers must handle partial output.

Source

Thrown at litellm/llms/oci/chat/cohere.py:284

    ``prior_tool_calls_emitted`` lets the caller signal whether tool calls
    were already emitted in earlier chunks of the same stream. When set, the
    terminal consolidation chunk's tool calls are suppressed (they would
    duplicate prior deltas); otherwise they are passed through so a stream
    that delivers tool calls only on the terminal chunk doesn't silently
    drop them.

    ``prior_text_emitted`` plays the analogous role for the ``text`` field:
    when set, the terminal consolidation chunk's ``text`` is suppressed
    (it would re-emit the full assembled response on top of prior deltas);
    when unset (e.g. a degenerate stream that delivers the entire response
    in a single SSE event carrying both ``chatHistory`` and ``finishReason``),
    the text is passed through so the response content isn't silently lost.
    """
    try:
        typed_chunk: Final = CohereStreamChunk(**dict_chunk)
    except (TypeError, ValidationError) as e:
        raise OCIError(
            status_code=500,
            message=f"Chunk cannot be parsed as CohereStreamChunk: {e}",
        )

    if typed_chunk.index is None:
        typed_chunk.index = 0

    # OCI Cohere's terminal SSE event re-sends the full assembled response in
    # `text` alongside a populated `chatHistory` and a non-null `finishReason`.
    # Emitting that text would concatenate the whole response onto the
    # already-streamed deltas. We require both signals to be present so that a
    # future API change which adds `chatHistory` to intermediate chunks (or a
    # rare early-populated case) doesn't silently drop legitimate token deltas.
    is_terminal_consolidation: Final = typed_chunk.chatHistory is not None and typed_chunk.finishReason is not None
    # On non-terminal text-free chunks (e.g. tool-call-only or keep-alive
    # chunks) emit ``content=None`` rather than ``content=""`` so downstream
    # stream-mergers that distinguish "no text in this delta" from "an
    # explicitly empty text delta" behave correctly.

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Upgrade litellm — streaming chunk models are updated as OCI event types evolve.
2. Capture and inspect the failing SSE line: log raw chunk deltas before the exception to see which event breaks parsing.
3. Bypass intermediaries that rewrite SSE (buffering proxies, custom middleware) when testing, to isolate where the corruption is introduced.
4. If it reproduces on the latest litellm, report it with the raw offending chunk so CohereStreamChunk can be loosened.

Example fix

# before
for event in litellm.completion(model='oci/cohere.command-r-plus', messages=msgs, stream=True):
    ...
# a malformed SSE frame aborts the loop with OCIError(500)

# after
from litellm.exceptions import APIError
try:
    for event in litellm.completion(..., stream=True):
        ...
except APIError:
    log.warning('stream aborted; keeping partial output')  # decide on partial-result policy

collected = []
try:
    for event in litellm.completion(model='oci/cohere.command-r-plus', messages=msgs, stream=True):
        collected.append(event.choices[0].delta.content or '')
except OCIError as e:
    if 'CohereStreamChunk' in str(e):
        logger.warning('stream broke mid-flight; using partial output (%d chars)', sum(map(len, collected)))
        # policy decision: serve partial text, or retry non-streaming once
        text = ''.join(collected)

When it happens

Trigger: Streaming completion on an OCI Cohere model when one SSE event has an unexpected shape: a new event type (usage/keep-alive/error frame), a field type change (text as non-string, toolCalls as wrong shape), or a truncated final event from a proxy that buffers/rewrites SSE.

Common situations: OCI introduces a new SSE event type that the adapter model rejects; a load balancer or middleware injects malformed SSE frames; long streams where a mid-stream schema change or partial flush produces an incomplete JSON object; version drift between an older litellm adapter and a newer OCI streaming API.

Related errors

• Failed to decode JSON from chunk: {chunk}
• Response cannot be casted to CohereChatResult: {e}
• Chunk is not a string: {chunk}
• Chunk does not start with 'data:': {chunk}
• Chunk cannot be parsed as JSON: {e}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/c5da6c590ccc0aff. Report an issue: GitHub.