{"record":{"id":"629b3ee9f352a2de","repo":"BerriAI/litellm","slug":"chunk-cannot-be-parsed-as-json-e","errorCode":null,"errorMessage":"Chunk cannot be parsed as JSON: {e}","messagePattern":"Chunk cannot be parsed as JSON: (.+?)","errorType":"exception","errorClass":"OCIError","httpStatus":500,"severity":"error","filePath":"litellm/llms/oci/chat/transformation.py","lineNumber":754,"sourceCode":"        # tool calls. The Cohere handler uses this to decide whether the\n        # terminal consolidation chunk's tool calls are duplicates (suppress)\n        # or the only copy of the tool calls (pass through).\n        self._cohere_tool_calls_emitted = False\n        # Analogous flag for text content. Lets the Cohere handler distinguish\n        # the common case (prior deltas already streamed the text, so the\n        # terminal chunk's text is a duplicate to suppress) from the degenerate\n        # single-event case (terminal chunk carries the only copy of the text).\n        self._cohere_text_emitted = False\n\n    def chunk_creator(self, chunk: Any) -> ModelResponseStream:\n        if not isinstance(chunk, str):\n            raise ValueError(f\"Chunk is not a string: {chunk}\")\n        if not chunk.startswith(\"data:\"):\n            raise ValueError(f\"Chunk does not start with 'data:': {chunk}\")\n        try:\n            dict_chunk: Final = json.loads(chunk[5:])\n        except json.JSONDecodeError as e:\n            raise OCIError(\n                status_code=500,\n                message=f\"Chunk cannot be parsed as JSON: {e}\",\n            )\n\n        if dict_chunk.get(\"apiFormat\") == \"COHERE\":\n            result: Final = handle_cohere_stream_chunk(\n                dict_chunk,\n                prior_tool_calls_emitted=self._cohere_tool_calls_emitted,\n                prior_text_emitted=self._cohere_text_emitted,\n            )\n            if not self._cohere_tool_calls_emitted:\n                for choice in result.choices:\n                    if getattr(choice.delta, \"tool_calls\", None) is not None:\n                        self._cohere_tool_calls_emitted = True\n                        break\n            if not self._cohere_text_emitted:\n                for choice in result.choices:\n                    if getattr(choice.delta, \"content\", None):","sourceCodeStart":736,"sourceCodeEnd":772,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/oci/chat/transformation.py#L736-L772","documentation":"After stripping the 'data:' prefix, chunk_creator json.loads the remainder; if it is not valid JSON the handler raises OCIError(500) with the JSONDecodeError detail. This points to a truncated or corrupted SSE payload rather than a request problem — the connection delivered bytes that do not form a complete JSON document.","triggerScenarios":"SSE line split mid-JSON by a buffering intermediary, a network drop producing a partial final event, or an upstream OCI gateway emitting a malformed event. Chunk strings like 'data:{\"apiFormat\": ' (cut off) trigger it.","commonSituations":"Load balancers/proxies with aggressive flush intervals truncating event frames; connections killed by idle timeouts leaving half-delivered events; very large tool-call deltas exceeding an intermediary's line-buffer limit. The status_code=500 here is synthetic — it represents a local parse failure, not an OCI HTTP status.","solutions":["Bypass proxies/LBs or configure them for unbuffered SSE passthrough (disable gzip, set flush interval to 0).","Retry the request — truncation from transient network drops is not deterministic.","If reproducible with curl -N, capture the raw stream and inspect the malformed event to identify which hop corrupts it.","Reduce max_tokens or tool-call size if corruption correlates with very large single events."],"exampleFix":"# before\nstream = litellm.completion(model=\"oci/...\", messages=m, stream=True)\nfor ev in stream:\n    pass  # mid-stream OCIError(500) on truncated JSON\n\n# after — treat parse failure as retryable stream error\nfrom litellm.llms.oci.common_utils import OCIError\ntry:\n    for ev in litellm.completion(model=\"oci/...\", messages=m, stream=True):\n        pass\nexcept OCIError as e:\n    if \"parsed as JSON\" in str(e):\n        retry_with_backoff()","handlingStrategy":"retry","validationCode":"import json\n\ndef is_parseable_sse(chunk: str) -> bool:\n    try:\n        json.loads(chunk[5:])\n        return True\n    except json.JSONDecodeError:\n        return False","typeGuard":null,"tryCatchPattern":"from litellm.llms.oci.common_utils import OCIError\ntry:\n    for ev in stream:\n        handle(ev)\nexcept OCIError as e:\n    if \"parsed as JSON\" in str(e):\n        restart_stream_from_last_checkpoint()  # retry, partial output already consumed\n    raise","preventionTips":["Ensure proxies pass SSE through unbuffered (no gzip, flush immediately).","Persist partial results while streaming so a parse failure is recoverable.","Treat mid-stream parse failures as transient and retryable."],"tags":["oci","streaming","sse","json","parsing"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}