BerriAI/litellm · error · ValueError

Failed to convert ModelResponse to ModelResponseStream: {mod

Error message

Failed to convert ModelResponse to ModelResponseStream: {model_response}. Error: {e}

What it means

When a provider returns a complete (non-streaming) ModelResponse that LiteLLM must fake as a stream, the generic converter builds a chat.completion.chunk from the response's fields. If any field access or pydantic construction fails (malformed choices, missing attributes on a partial response), the whole conversion is caught and re-raised with the serialized response for debugging.

Source

Thrown at litellm/llms/base_llm/base_model_iterator.py:63

                    ),
                    finish_reason=choice.finish_reason,
                )
            )
        processed_chunk: Final = ModelResponseStream(
            id=model_response.id,
            object="chat.completion.chunk",
            created=model_response.created,
            model=model_response.model,
            choices=streaming_choices,
        )
        # Carry usage onto the streaming chunk so fake-streamed responses
        # (e.g. Vertex AI Gemma :predict) still report token counts.
        usage: Final = getattr(model_response, "usage", None)
        if usage is not None:
            setattr(processed_chunk, "usage", usage)
        return processed_chunk
    except Exception as e:
        raise ValueError(f"Failed to convert ModelResponse to ModelResponseStream: {model_response}. Error: {e}")


class BaseModelResponseIterator:
    def __init__(self, streaming_response, sync_stream: bool, json_mode: bool | None = False):
        self.streaming_response = streaming_response
        self.response_iterator = self.streaming_response
        self.json_mode = json_mode
        self.http_response: httpx.Response | None = None

    async def aclose(self) -> None:
        """Close the upstream HTTP response so the provider connection is
        released (and a backend like vLLM aborts generation) when the stream
        is abandoned before its natural end.

        ``streaming_response`` is usually a bare ``aiter_lines()`` generator
        that holds no reference to the response, so the handler that owns the
        response attaches it here after construction."""
        if self.http_response is not None:

View on GitHub (pinned to 6c2dcb801b)

Solutions

  1. Inspect the printed model_response in the message — usually a malformed choices list is visible
  2. If building ModelResponse yourself, construct it via ModelResponse(choices=[Choices(delta=..., message=...)]) so fields are well-formed
  3. Call with stream=False to see the raw response error directly instead of the conversion failure
  4. Upgrade litellm — conversion helpers gain field tolerance over versions

Example fix

# before (custom handler)
resp = ModelResponse(); resp.choices = [{}]  # malformed

# after
from litellm.types.utils import ModelResponse, Choices, Delta
resp = ModelResponse(choices=[Choices(index=0, delta=Delta(content='hi'), finish_reason='stop')])
Defensive patterns

Strategy: try-catch

Try / catch

try:
    for chunk in response:
        process(chunk)
except ValueError as e:
    if 'Failed to convert ModelResponse to ModelResponseStream' in str(e):
        log.error('malformed provider response: %s', e)
        # fall back to non-streaming call
    else:
        raise

Prevention

When it happens

Trigger: A provider handler returns a ModelResponse whose choices/usage are malformed or whose types don't fit Choice chunk construction (e.g. custom provider adapter or a mocked response), and stream=True forces the conversion path.

Common situations: Writing/testing a custom provider integration that returns hand-built ModelResponse objects; a mocked response object missing expected attributes; version mismatches between litellm core and a provider adapter after upgrade.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/9f854ba90da6cb90. Report an issue: GitHub.