{"record":{"id":"c5da6c590ccc0aff","repo":"BerriAI/litellm","slug":"chunk-cannot-be-parsed-as-coherestreamchunk-e","errorCode":null,"errorMessage":"Chunk cannot be parsed as CohereStreamChunk: {e}","messagePattern":"Chunk cannot be parsed as CohereStreamChunk: (.+?)","errorType":"http","errorClass":"OCIError","httpStatus":500,"severity":"error","filePath":"litellm/llms/oci/chat/cohere.py","lineNumber":284,"sourceCode":"\n    ``prior_tool_calls_emitted`` lets the caller signal whether tool calls\n    were already emitted in earlier chunks of the same stream. When set, the\n    terminal consolidation chunk's tool calls are suppressed (they would\n    duplicate prior deltas); otherwise they are passed through so a stream\n    that delivers tool calls only on the terminal chunk doesn't silently\n    drop them.\n\n    ``prior_text_emitted`` plays the analogous role for the ``text`` field:\n    when set, the terminal consolidation chunk's ``text`` is suppressed\n    (it would re-emit the full assembled response on top of prior deltas);\n    when unset (e.g. a degenerate stream that delivers the entire response\n    in a single SSE event carrying both ``chatHistory`` and ``finishReason``),\n    the text is passed through so the response content isn't silently lost.\n    \"\"\"\n    try:\n        typed_chunk: Final = CohereStreamChunk(**dict_chunk)\n    except (TypeError, ValidationError) as e:\n        raise OCIError(\n            status_code=500,\n            message=f\"Chunk cannot be parsed as CohereStreamChunk: {e}\",\n        )\n\n    if typed_chunk.index is None:\n        typed_chunk.index = 0\n\n    # OCI Cohere's terminal SSE event re-sends the full assembled response in\n    # `text` alongside a populated `chatHistory` and a non-null `finishReason`.\n    # Emitting that text would concatenate the whole response onto the\n    # already-streamed deltas. We require both signals to be present so that a\n    # future API change which adds `chatHistory` to intermediate chunks (or a\n    # rare early-populated case) doesn't silently drop legitimate token deltas.\n    is_terminal_consolidation: Final = typed_chunk.chatHistory is not None and typed_chunk.finishReason is not None\n    # On non-terminal text-free chunks (e.g. tool-call-only or keep-alive\n    # chunks) emit ``content=None`` rather than ``content=\"\"`` so downstream\n    # stream-mergers that distinguish \"no text in this delta\" from \"an\n    # explicitly empty text delta\" behave correctly.","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/oci/chat/cohere.py#L266-L302","documentation":"During streaming, each SSE chunk from OCI Cohere is parsed into the `CohereStreamChunk` Pydantic model. A chunk whose fields do not fit raises OCIError(500) 'Chunk cannot be parsed as CohereStreamChunk: {details}'. Unlike the non-streaming variant this always uses status 500, and it can fire mid-stream after valid chunks were already consumed — callers must handle partial output.","triggerScenarios":"Streaming completion on an OCI Cohere model when one SSE event has an unexpected shape: a new event type (usage/keep-alive/error frame), a field type change (text as non-string, toolCalls as wrong shape), or a truncated final event from a proxy that buffers/rewrites SSE.","commonSituations":"OCI introduces a new SSE event type that the adapter model rejects; a load balancer or middleware injects malformed SSE frames; long streams where a mid-stream schema change or partial flush produces an incomplete JSON object; version drift between an older litellm adapter and a newer OCI streaming API.","solutions":["Upgrade litellm — streaming chunk models are updated as OCI event types evolve.","Capture and inspect the failing SSE line: log raw chunk deltas before the exception to see which event breaks parsing.","Bypass intermediaries that rewrite SSE (buffering proxies, custom middleware) when testing, to isolate where the corruption is introduced.","If it reproduces on the latest litellm, report it with the raw offending chunk so CohereStreamChunk can be loosened."],"exampleFix":"# before\nfor event in litellm.completion(model='oci/cohere.command-r-plus', messages=msgs, stream=True):\n    ...\n# a malformed SSE frame aborts the loop with OCIError(500)\n\n# after\nfrom litellm.exceptions import APIError\ntry:\n    for event in litellm.completion(..., stream=True):\n        ...\nexcept APIError:\n    log.warning('stream aborted; keeping partial output')  # decide on partial-result policy","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"collected = []\ntry:\n    for event in litellm.completion(model='oci/cohere.command-r-plus', messages=msgs, stream=True):\n        collected.append(event.choices[0].delta.content or '')\nexcept OCIError as e:\n    if 'CohereStreamChunk' in str(e):\n        logger.warning('stream broke mid-flight; using partial output (%d chars)', sum(map(len, collected)))\n        # policy decision: serve partial text, or retry non-streaming once\n        text = ''.join(collected)","preventionTips":["Accumulate streamed text incrementally so a mid-stream failure still leaves usable partial output.","Offer a non-streaming retry path for streams that die on chunk parsing.","Keep litellm current: streaming chunk models track new OCI SSE event types."],"tags":["oci","cohere","streaming","sse","response-parsing"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}