BerriAI/litellm · error · ValueError

Failed to decode JSON from chunk: {chunk}

Error message

Failed to decode JSON from chunk: {chunk}

What it means

Raised by the Databricks streaming chunk parser when json.loads (or SSE parsing upstream of it) hits invalid JSON in a chunk. The offending raw chunk text is included in the ValueError message. Note this fires from within chunk parsing, so it usually surfaces wrapped by the iterator's RuntimeError handler.

Source

Thrown at litellm/llms/databricks/streaming_utils.py:90

            usage_chunk: Final[Usage | None] = getattr(processed_chunk, "usage", None)
            if usage_chunk is not None:
                usage = ChatCompletionUsageBlock(
                    prompt_tokens=usage_chunk.prompt_tokens,
                    completion_tokens=usage_chunk.completion_tokens,
                    total_tokens=usage_chunk.total_tokens,
                )

            return GenericStreamingChunk(
                text=text,
                tool_use=tool_use,
                is_finished=is_finished,
                finish_reason=finish_reason,
                usage=usage,
                index=0,
            )
        except json.JSONDecodeError:
            raise ValueError(f"Failed to decode JSON from chunk: {chunk}")

    # Sync iterator
    def __iter__(self):
        self.response_iterator = self.streaming_response
        return self

    def __next__(self):
        if not hasattr(self, "response_iterator"):
            self.response_iterator = self.streaming_response
        try:
            chunk = self.response_iterator.__next__()
        except StopIteration:
            raise StopIteration
        except ValueError as e:
            raise RuntimeError(f"Error receiving chunk from stream: {e}")

        try:
            chunk = litellm.CustomStreamWrapper._strip_sse_data_from_chunk(chunk) or ""

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Log the chunk from the message to identify what non-JSON content is arriving
2. If proxy-related, disable response buffering / adjust idle timeouts on the intermediate proxy
3. Retry the request — truncation from transient network faults is often intermittent
4. If the endpoint emits keep-alive text, fix the model server to use proper SSE format

try:
    for chunk in stream:
        handle(chunk)
except (ValueError, RuntimeError) as e:
    if "Failed to decode JSON" in str(e):
        logger.warning("malformed SSE chunk, retrying stream")
        stream = litellm.completion(**params, stream=True)
        for chunk in stream:
            handle(chunk)
    else:
        raise

When it happens

Trigger: A streamed SSE data line contains malformed JSON: truncated line due to network interruption, a chunk split incorrectly across reads, or an endpoint emitting non-JSON keep-alive/debug lines as data payloads.

Common situations: Long streams dropped mid-chunk by proxies with response buffering limits; custom Databricks model servers logging plain-text lines into the stream; flaky connections between the proxy and Databricks.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Mavvrik FOCUS destination: response missing 'url' field: {re
• litellm.MidStreamFallbackError: {message}
• Failed to decode JSON from chunk: {chunk}
• Failed to parse raw Azure embedding response: {json_error}
• Error receiving chunk from stream: {e}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/729fe053ff326dd2. Report an issue: GitHub.