BerriAI/litellm · error

Error receiving chunk from stream

Error message

Error receiving chunk from stream: {e}

What it means

The Databricks streaming iterator wraps the underlying HTTP response iterator: when response_iterator.__next__() raises ValueError (httpx raises it on decoding failures, e.g. content decoding errors or a body that violates Content-Length), it is converted to RuntimeError('Error receiving chunk from stream: ...'). This is a transport-level failure of the byte stream mid-iteration, not a schema problem — the stream is dead and iteration stops. The async path wraps all exceptions the same way.

Solutions

  1. Inspect the wrapped exception text after 'stream:' — it names the underlying transport failure (decode error, content-length mismatch)
  2. Retry the request with a fresh connection; transient stream corruption usually clears
  3. If it recurs on the same endpoint, check proxy/LB idle timeouts and the endpoint's response headers (Content-Length, Content-Encoding, charset)
  4. For very long completions, reduce max_tokens or chunk the work so streams are shorter-lived

Example fix

# before
for part in litellm.completion(model='databricks/my-model', messages=msgs, stream=True):
    print(part)  # RuntimeError kills iteration mid-stream

# after — retry streaming once on transport failure
from tenacity import retry, retry_if_exception, stop_after_attempt

@retry(retry=retry_if_exception(lambda e: 'Error receiving chunk from stream' in str(e)),
       stop=stop_after_attempt(2))
def stream_completion():
    return litellm.completion(model='databricks/my-model', messages=msgs, stream=True)

for part in stream_completion():
    print(part)
Defensive patterns

Strategy: retry

Try / catch

def stream_with_retry(model, messages, attempts=2):
    for i in range(attempts):
        try:
            return litellm.completion(model=model, messages=messages, stream=True)
        except RuntimeError as e:
            if i == attempts - 1 or 'Error receiving chunk from stream' not in str(e):
                raise
    # caller re-iterates; partial output is lost, retry restarts the completion

stream = stream_with_retry('databricks/my-ep', msgs)
for chunk in stream:
    handle(chunk)

Prevention

When it happens

Trigger: Streaming a Databricks completion when the connection degrades mid-stream: encoding mismatch ('Unable to decode content with encoding ...'), truncated body (content shorter/longer than Content-Length), proxy closing or corrupting the connection, or any ValueError raised by the httpx line/byte iterator.

Common situations: Long streams through flaky proxies/VPNs; servers sending wrong Content-Length or charset headers; mobile/ephemeral networks dropping connections partway; load balancers with aggressive idle timeouts.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/0c2f16828cc54647. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/databricks/streaming_utils.py:105

                index=0,
            )
        except json.JSONDecodeError:
            raise ValueError(f"Failed to decode JSON from chunk: {chunk}")

    # Sync iterator
    def __iter__(self):
        self.response_iterator = self.streaming_response
        return self

    def __next__(self):
        if not hasattr(self, "response_iterator"):
            self.response_iterator = self.streaming_response
        try:
            chunk = self.response_iterator.__next__()
        except StopIteration:
            raise StopIteration
        except ValueError as e:
            raise RuntimeError(f"Error receiving chunk from stream: {e}")

        try:
            chunk = litellm.CustomStreamWrapper._strip_sse_data_from_chunk(chunk) or ""
            chunk = chunk.strip()
            if len(chunk) > 0:
                json_chunk: Final = json.loads(chunk)
                return self.chunk_parser(chunk=json_chunk)
            else:
                return GenericStreamingChunk(
                    text="",
                    is_finished=False,
                    finish_reason="",
                    usage=None,
                    index=0,
                    tool_use=None,
                )
        except StopIteration:
            raise StopIteration

View on GitHub (pinned to 77b7c6c40c)