BerriAI/litellm · error · Exception

{self.custom_llm_provider} raised a streaming error - finish

Error message

{self.custom_llm_provider} raised a streaming error - finish_reason: error, no content string given. Received Chunk={response_obj}

What it means

Raised while processing a streaming response: the provider sent a chunk with finish_reason == 'error' and no content string. LiteLLM's streaming handler treats an error finish reason as a hard failure because it cannot distinguish a provider-side abort from a normal completion, and it embeds the raw Chunk in the message for diagnosis.

Source

Thrown at litellm/litellm_core_utils/streaming_handler.py:1436

                self.system_fingerprint = chunk.system_fingerprint
            if response_obj["is_finished"]:
                self.received_finish_reason = response_obj["finish_reason"]
        else:  # openai / azure chat model
            if self.custom_llm_provider in [
                LlmProviders.AZURE.value,
                LlmProviders.AZURE_AI.value,
            ]:
                if isinstance(chunk, BaseModel) and hasattr(chunk, "model"):
                    # for azure, we need to pass the model from the original chunk
                    self.model = getattr(chunk, "model", self.model)
            response_obj = self.handle_openai_chat_completion_chunk(chunk)
            if response_obj is None:
                return _ProviderChunkEarlyReturn(None)
            completion_obj["content"] = response_obj["text"]
            self.intermittent_finish_reason = response_obj.get("finish_reason", None)
            if response_obj["is_finished"]:
                if response_obj["finish_reason"] == "error":
                    raise Exception(
                        f"{self.custom_llm_provider} raised a streaming error - finish_reason: error, no content string given. Received Chunk={response_obj}"
                    )
                self.received_finish_reason = response_obj["finish_reason"]
            if response_obj.get("original_chunk", None) is not None:
                if hasattr(response_obj["original_chunk"], "id"):
                    model_response = self.set_model_id(response_obj["original_chunk"].id, model_response)
                if hasattr(response_obj["original_chunk"], "system_fingerprint"):
                    model_response.system_fingerprint = response_obj["original_chunk"].system_fingerprint
                    self.system_fingerprint = response_obj["original_chunk"].system_fingerprint
            if response_obj["logprobs"] is not None:
                model_response.choices[0].logprobs = response_obj["logprobs"]

            if response_obj["usage"] is not None:
                if isinstance(response_obj["usage"], dict):
                    setattr(
                        model_response,
                        "usage",
                        litellm.Usage(

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Retry the request (with or without stream=True) - most occurrences are transient provider-side aborts; check the provider status page.
2. Inspect the full Received Chunk= payload in the exception for a provider error code (e.g. content_filter) and fix the prompt/model config accordingly.
3. If behind an OpenAI-compatible proxy, check or upgrade it - some gateways emit finish_reason='error' on timeout; raise their timeout.
4. If it recurs on one deployment, route around it with litellm.Router fallback deployments and num_retries>=1.

Example fix

# before
resp = await litellm.acompletion(model="azure/gpt-4o", messages=msgs, stream=True)
async for e in resp: process(e)  # raises mid-iteration

# after
from litellm import Router
router = Router(model_list=[
  {"model_name": "gpt-4o", "litellm_params": {"model": "azure/gpt-4o-primary"}},
  {"model_name": "gpt-4o", "litellm_params": {"model": "azure/gpt-4o-backup"}},
], num_retries=2, timeout=120)
resp = await router.acompletion(model="gpt-4o", messages=msgs, stream=True)

try:
    async for event in stream: process(event)
except Exception as e:  # handler raises a plain Exception
    if "raised a streaming error" in str(e):
        resp = await router.acompletion(...)  # retry once, non-stream or fallback model
    else:
        raise

When it happens

Trigger: A streaming completion() call (stream=True) where the upstream LLM provider terminates the stream with finish_reason 'error' - e.g. Azure/OpenAI-compatible gateways, Vertex, or Bedrock emitting an error termination frame mid-stream, or content-filter terminations.

Common situations: Long streaming generations cut off server-side; Azure OpenAI content-moderation kills; unstable OpenAI-compatible proxies (vLLM, self-hosted gateways) that map internal errors to finish_reason='error'; transient provider incidents under load.

Related errors

• litellm.MidStreamFallbackError: {message}
• Braintrust API error: {e.response.text}
• Failed to connect to Braintrust API: {str(e)}
• api_base is required for Pydantic AI agents
• Redis circuit breaker is open — skipping {name}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/65169e04a4f3f21c. Report an issue: GitHub.