{"record":{"id":"729fe053ff326dd2","repo":"BerriAI/litellm","slug":"failed-to-decode-json-from-chunk-chunk-729fe0","errorCode":null,"errorMessage":"Failed to decode JSON from chunk: {chunk}","messagePattern":"Failed to decode JSON from chunk: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/databricks/streaming_utils.py","lineNumber":90,"sourceCode":"\n            usage_chunk: Final[Usage | None] = getattr(processed_chunk, \"usage\", None)\n            if usage_chunk is not None:\n                usage = ChatCompletionUsageBlock(\n                    prompt_tokens=usage_chunk.prompt_tokens,\n                    completion_tokens=usage_chunk.completion_tokens,\n                    total_tokens=usage_chunk.total_tokens,\n                )\n\n            return GenericStreamingChunk(\n                text=text,\n                tool_use=tool_use,\n                is_finished=is_finished,\n                finish_reason=finish_reason,\n                usage=usage,\n                index=0,\n            )\n        except json.JSONDecodeError:\n            raise ValueError(f\"Failed to decode JSON from chunk: {chunk}\")\n\n    # Sync iterator\n    def __iter__(self):\n        self.response_iterator = self.streaming_response\n        return self\n\n    def __next__(self):\n        if not hasattr(self, \"response_iterator\"):\n            self.response_iterator = self.streaming_response\n        try:\n            chunk = self.response_iterator.__next__()\n        except StopIteration:\n            raise StopIteration\n        except ValueError as e:\n            raise RuntimeError(f\"Error receiving chunk from stream: {e}\")\n\n        try:\n            chunk = litellm.CustomStreamWrapper._strip_sse_data_from_chunk(chunk) or \"\"","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/databricks/streaming_utils.py#L72-L108","documentation":"Raised by the Databricks streaming chunk parser when json.loads (or SSE parsing upstream of it) hits invalid JSON in a chunk. The offending raw chunk text is included in the ValueError message. Note this fires from within chunk parsing, so it usually surfaces wrapped by the iterator's RuntimeError handler.","triggerScenarios":"A streamed SSE data line contains malformed JSON: truncated line due to network interruption, a chunk split incorrectly across reads, or an endpoint emitting non-JSON keep-alive/debug lines as data payloads.","commonSituations":"Long streams dropped mid-chunk by proxies with response buffering limits; custom Databricks model servers logging plain-text lines into the stream; flaky connections between the proxy and Databricks.","solutions":["Log the chunk from the message to identify what non-JSON content is arriving","If proxy-related, disable response buffering / adjust idle timeouts on the intermediate proxy","Retry the request — truncation from transient network faults is often intermittent","If the endpoint emits keep-alive text, fix the model server to use proper SSE format"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    for chunk in stream:\n        handle(chunk)\nexcept (ValueError, RuntimeError) as e:\n    if \"Failed to decode JSON\" in str(e):\n        logger.warning(\"malformed SSE chunk, retrying stream\")\n        stream = litellm.completion(**params, stream=True)\n        for chunk in stream:\n            handle(chunk)\n    else:\n        raise","preventionTips":["Disable response buffering (e.g. nginx proxy_buffering off; X-Accel-Buffering: no) on SSE routes","Set adequate read/idle timeouts on proxies between client and Databricks","Log raw unparsable chunks to identify keep-alive or debug lines from the model server"],"tags":["databricks","streaming","json","network"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}