{"record":{"id":"c5da6c590ccc0aff","repo":"BerriAI/litellm","slug":"chunk-cannot-be-parsed-as-coherestreamchunk-e","errorCode":null,"errorMessage":"Chunk cannot be parsed as CohereStreamChunk: {e}","messagePattern":"Chunk cannot be parsed as CohereStreamChunk: (.+?)","errorType":"exception","errorClass":"OCIError","httpStatus":500,"severity":"error","filePath":"litellm/llms/oci/chat/cohere.py","lineNumber":284,"sourceCode":"\n    ``prior_tool_calls_emitted`` lets the caller signal whether tool calls\n    were already emitted in earlier chunks of the same stream. When set, the\n    terminal consolidation chunk's tool calls are suppressed (they would\n    duplicate prior deltas); otherwise they are passed through so a stream\n    that delivers tool calls only on the terminal chunk doesn't silently\n    drop them.\n\n    ``prior_text_emitted`` plays the analogous role for the ``text`` field:\n    when set, the terminal consolidation chunk's ``text`` is suppressed\n    (it would re-emit the full assembled response on top of prior deltas);\n    when unset (e.g. a degenerate stream that delivers the entire response\n    in a single SSE event carrying both ``chatHistory`` and ``finishReason``),\n    the text is passed through so the response content isn't silently lost.\n    \"\"\"\n    try:\n        typed_chunk: Final = CohereStreamChunk(**dict_chunk)\n    except (TypeError, ValidationError) as e:\n        raise OCIError(\n            status_code=500,\n            message=f\"Chunk cannot be parsed as CohereStreamChunk: {e}\",\n        )\n\n    if typed_chunk.index is None:\n        typed_chunk.index = 0\n\n    # OCI Cohere's terminal SSE event re-sends the full assembled response in\n    # `text` alongside a populated `chatHistory` and a non-null `finishReason`.\n    # Emitting that text would concatenate the whole response onto the\n    # already-streamed deltas. We require both signals to be present so that a\n    # future API change which adds `chatHistory` to intermediate chunks (or a\n    # rare early-populated case) doesn't silently drop legitimate token deltas.\n    is_terminal_consolidation: Final = typed_chunk.chatHistory is not None and typed_chunk.finishReason is not None\n    # On non-terminal text-free chunks (e.g. tool-call-only or keep-alive\n    # chunks) emit ``content=None`` rather than ``content=\"\"`` so downstream\n    # stream-mergers that distinguish \"no text in this delta\" from \"an\n    # explicitly empty text delta\" behave correctly.","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/oci/chat/cohere.py#L266-L302","documentation":"While streaming an OCI Cohere completion, each SSE event is parsed into the pydantic model CohereStreamChunk. If a chunk's JSON does not validate (TypeError/ValidationError), LiteLLM raises OCIError (500) and aborts the stream mid-iteration. Like 1682 this indicates the streamed chunk shape diverges from what this LiteLLM version expects — a server-side/packaging mismatch, not malformed caller input.","triggerScenarios":"litellm.completion(..., stream=True) with an oci/cohere.* model where an SSE event fails CohereStreamChunk validation — e.g. OCI emits a new keep-alive/event type, a field changes type between intermediate and terminal chunks, or an error event arrives in the SSE channel.","commonSituations":"OCI rolls out stream-format changes per region or model version; dedicated-AI-cluster firmware serving an older/newer SSE contract than public OCI; old litellm version against updated endpoints; sporadic proxy/LB-injected non-JSON lines into the SSE channel.","solutions":["Upgrade LiteLLM to the latest release — streaming chunk parsers are patched as OCI formats evolve","Reproduce with stream=False first: if the non-streaming call works, the issue is specific to the SSE chunk shape; capture chunks (litellm.client or httpx event hook) and compare against the embedded pydantic error","Retry the request — transient corrupt SSE frames (network truncation) also land here; a fresh connection usually succeeds","Try a different OCI region or cohere model version to sidestep a model-specific stream quirk","If persistent, file a litellm issue with the offending sanitized chunk; meanwhile pin the last known-good litellm version"],"exampleFix":"# before\nstream = litellm.completion(model=\"oci/cohere.command-r-16k\", messages=msgs, stream=True)\nfor chunk in stream:  # OCIError: Chunk cannot be parsed as CohereStreamChunk\n    ...\n\n# after: guard the iteration and fall back to non-streaming\nfrom litellm.exceptions import OCIError\ntry:\n    for chunk in litellm.completion(model=\"oci/cohere.command-r-16k\", messages=msgs, stream=True):\n        print(chunk.choices[0].delta.content or \"\", end=\"\")\nexcept OCIError:\n    resp = litellm.completion(model=\"oci/cohere.command-r-16k\", messages=msgs)  # no stream\n    print(resp.choices[0].message.content)","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"from litellm.exceptions import OCIError\n\ntry:\n    text = \"\"\n    for chunk in litellm.completion(model=MODEL, messages=msgs, stream=True):\n        text += chunk.choices[0].delta.content or \"\"\nexcept OCIError as e:\n    if \"CohereStreamChunk\" not in str(e):\n        raise\n    resp = litellm.completion(model=MODEL, messages=msgs)  # degrade to non-streaming\n    text = resp.choices[0].message.content","preventionTips":["Canary-test streaming against each OCI model/region in staging before prod rollout","Build streams with a non-stream fallback path so SSE parser drift cannot hard-fail UX","Retry once on chunk-parse failures — truncated SSE frames are frequently transient"],"tags":["oci","cohere","streaming","sse","pydantic","schema-drift"],"backgroundTag":"response-schema-validation-failed","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}