{"record":{"id":"951b6211e33540e9","repo":"BerriAI/litellm","slug":"failed-to-decode-json-from-chunk-chunk","errorCode":null,"errorMessage":"Failed to decode JSON from chunk: {chunk}","messagePattern":"Failed to decode JSON from chunk: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/anthropic/chat/handler.py","lineNumber":977,"sourceCode":"                        index=index,\n                        delta=Delta(\n                            content=text,\n                            tool_calls=[tool_use] if tool_use is not None else None,\n                            provider_specific_fields=(provider_specific_fields if provider_specific_fields else None),\n                            thinking_blocks=(thinking_blocks if thinking_blocks else None),\n                            reasoning_content=reasoning_content,\n                        ),\n                        finish_reason=finish_reason,\n                    )\n                ],\n                usage=usage,\n                id=self.response_id,\n            )\n\n            return returned_chunk\n\n        except json.JSONDecodeError:\n            raise ValueError(f\"Failed to decode JSON from chunk: {chunk}\")\n\n    def _handle_json_mode_chunk(\n        self, text: str, tool_use: ChatCompletionToolCallChunk | None\n    ) -> tuple[str, ChatCompletionToolCallChunk | None]:\n        \"\"\"\n        If JSON mode is enabled, convert the tool call to a message.\n\n        Anthropic returns the JSON schema as part of the tool call\n        OpenAI returns the JSON schema as part of the content, this handles placing it in the content\n\n        Tool streaming follows Anthropic's fine-grained streaming pattern:\n        - content_block_start: Contains complete tool info (id, name, empty arguments)\n        - content_block_delta: Contains argument deltas (partial_json)\n        - content_block_stop: Signals end of tool\n\n        Reference: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/fine-grained-tool-streaming\n\n        Args:","sourceCodeStart":959,"sourceCodeEnd":995,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/anthropic/chat/handler.py#L959-L995","documentation":"Thrown by the Anthropic streaming chunk parser when json.loads fails on a chunk it was handed. LiteLLM's Anthropic streaming handler parses each SSE data payload as JSON before converting it to a GenericStreamingChunk; if the payload is not valid JSON, the json.JSONDecodeError is re-raised as this ValueError. It almost always means the bytes coming off the HTTP stream are not a well-formed Anthropic SSE JSON event.","triggerScenarios":"Iterating an anthropic/ streaming completion where a 'data:' payload contains malformed JSON: a proxy or gateway (e.g. nginx, Cloudflare, a custom LiteLLM proxy hop) that truncates or rewrites SSE frames; an upstream returning an HTML/text error page mid-stream; or accumulated partial JSON that was flushed without its remainder when the stream ended.","commonSituations":"Self-hosted LiteLLM proxy behind a buffering/rewriting proxy; Anthropic API base URL pointed at a third-party gateway that emits non-Anthropic SSE; truncation from aggressive read timeouts; occasionally a transient upstream 5xx body leaked into the stream.","solutions":["Reproduce with streaming disabled (stream=False) to see the raw body; if it is HTML/text, the problem is the endpoint or an intermediary, not your code.","Inspect the {chunk} text in the message to identify what actually arrived (HTML error page, truncated JSON, empty string).","If using a custom api_base or proxy, bypass it and call api.anthropic.com directly to confirm the proxy is mangling SSE frames.","Ensure SSE is not buffered/compressed by intermediaries (disable gzip on the proxy, set 'Cache-Control: no-cache', flush per event).","Upgrade LiteLLM — the accumulated-JSON flush logic in this iterator has had fixes for partial-JSON handling at stream end."],"exampleFix":"# before\nfor chunk in completion(stream=True):\n    process(chunk)  # ValueError kills the loop mid-stream\n\n# after\nfor chunk in completion(stream=True):\n    try:\n        process(chunk)\n    except ValueError as e:\n        if \"Failed to decode JSON from chunk\" in str(e):\n            log.warning(\"malformed SSE frame, skipping: %s\", e)\n            continue\n        raise","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    for chunk in stream:\n        consume(chunk)\nexcept ValueError as e:\n    if \"Failed to decode JSON\" in str(e):\n        log.warning(\"malformed SSE frame: %s\", e)\n        # keep partial content, do not retry the whole stream blindly\n    else:\n        raise","preventionTips":["Test streaming against api.anthropic.com directly before adding proxies.","Disable SSE buffering and compression on any intermediary.","Log raw frames at DEBUG level in staging to catch format drift early.","Keep LiteLLM updated for partial-JSON flush fixes."],"tags":["anthropic","streaming","sse","json","proxy"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}