{"record":{"id":"729fe053ff326dd2","repo":"BerriAI/litellm","slug":"failed-to-decode-json-from-chunk-chunk-729fe0","errorCode":null,"errorMessage":"Failed to decode JSON from chunk: {chunk}","messagePattern":"Failed to decode JSON from chunk: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"litellm/llms/databricks/streaming_utils.py","lineNumber":90,"sourceCode":"\n            usage_chunk: Final[Usage | None] = getattr(processed_chunk, \"usage\", None)\n            if usage_chunk is not None:\n                usage = ChatCompletionUsageBlock(\n                    prompt_tokens=usage_chunk.prompt_tokens,\n                    completion_tokens=usage_chunk.completion_tokens,\n                    total_tokens=usage_chunk.total_tokens,\n                )\n\n            return GenericStreamingChunk(\n                text=text,\n                tool_use=tool_use,\n                is_finished=is_finished,\n                finish_reason=finish_reason,\n                usage=usage,\n                index=0,\n            )\n        except json.JSONDecodeError:\n            raise ValueError(f\"Failed to decode JSON from chunk: {chunk}\")\n\n    # Sync iterator\n    def __iter__(self):\n        self.response_iterator = self.streaming_response\n        return self\n\n    def __next__(self):\n        if not hasattr(self, \"response_iterator\"):\n            self.response_iterator = self.streaming_response\n        try:\n            chunk = self.response_iterator.__next__()\n        except StopIteration:\n            raise StopIteration\n        except ValueError as e:\n            raise RuntimeError(f\"Error receiving chunk from stream: {e}\")\n\n        try:\n            chunk = litellm.CustomStreamWrapper._strip_sse_data_from_chunk(chunk) or \"\"","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/databricks/streaming_utils.py#L72-L108","documentation":"In Databricks streaming, each SSE data line is json.loads-ed in __next__ and handed to chunk_parser, which validates it by constructing litellm.ModelResponseStream(**chunk) inside a try block. If that construction fails with json.JSONDecodeError (malformed embedded JSON strings inside the chunk), it is re-raised as ValueError naming the chunk. Note: a plain undecodable SSE line is handled separately (logged, skipped as an empty chunk), so this error specifically means the dict decoded but its contents failed JSON validation.","triggerScenarios":"A Databricks stream chunk whose fields contain malformed JSON text (e.g. tool-call arguments or usage delivered as invalid JSON strings), or a chunk shape that breaks ModelResponseStream's parsing — from a non-OpenAI-compatible serving endpoint, a gateway mutating chunks, or a corrupted stream body.","commonSituations":"Custom/external model endpoints emitting non-standard chunk payloads; middleboxes rewriting SSE frames; schema drift between Databricks serving and LiteLLM's expected OpenAI-compatible chunk.","solutions":["Log the chunk embedded in the message and compare it field-by-field with the OpenAI streaming chunk schema","Confirm the endpoint is OpenAI-compatible (chat completions streaming shape) and reached via the expected databricks provider path","Retry with stream=False to see whether the non-streaming response from the same endpoint is well-formed","Upgrade LiteLLM; if the chunk is valid per Databricks docs, report it with the logged payload"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    for chunk in litellm.completion(model='databricks/my-ep', messages=msgs, stream=True):\n        handle(chunk)\nexcept Exception as e:\n    if 'Failed to decode JSON from chunk' in str(e):\n        log.error('malformed stream chunk: %s', str(e)[:2000])\n        resp = litellm.completion(model='databricks/my-ep', messages=msgs, stream=False)\n        handle(resp)\n    else:\n        raise","preventionTips":["Integration-test streaming against each endpoint before enabling it for traffic","Keep a non-streaming fallback for endpoints with non-standard chunk shapes"],"tags":["databricks","streaming","json","chunk-parsing"],"backgroundTag":"streaming-chunk-decode-error","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}