{"record":{"id":"9f854ba90da6cb90","repo":"BerriAI/litellm","slug":"failed-to-convert-modelresponse-to-modelresponsest","errorCode":null,"errorMessage":"Failed to convert ModelResponse to ModelResponseStream: {model_response}. Error: {e}","messagePattern":"Failed to convert ModelResponse to ModelResponseStream: (.+?)\\. Error: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/base_llm/base_model_iterator.py","lineNumber":63,"sourceCode":"                    ),\n                    finish_reason=choice.finish_reason,\n                )\n            )\n        processed_chunk: Final = ModelResponseStream(\n            id=model_response.id,\n            object=\"chat.completion.chunk\",\n            created=model_response.created,\n            model=model_response.model,\n            choices=streaming_choices,\n        )\n        # Carry usage onto the streaming chunk so fake-streamed responses\n        # (e.g. Vertex AI Gemma :predict) still report token counts.\n        usage: Final = getattr(model_response, \"usage\", None)\n        if usage is not None:\n            setattr(processed_chunk, \"usage\", usage)\n        return processed_chunk\n    except Exception as e:\n        raise ValueError(f\"Failed to convert ModelResponse to ModelResponseStream: {model_response}. Error: {e}\")\n\n\nclass BaseModelResponseIterator:\n    def __init__(self, streaming_response, sync_stream: bool, json_mode: bool | None = False):\n        self.streaming_response = streaming_response\n        self.response_iterator = self.streaming_response\n        self.json_mode = json_mode\n        self.http_response: httpx.Response | None = None\n\n    async def aclose(self) -> None:\n        \"\"\"Close the upstream HTTP response so the provider connection is\n        released (and a backend like vLLM aborts generation) when the stream\n        is abandoned before its natural end.\n\n        ``streaming_response`` is usually a bare ``aiter_lines()`` generator\n        that holds no reference to the response, so the handler that owns the\n        response attaches it here after construction.\"\"\"\n        if self.http_response is not None:","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/base_llm/base_model_iterator.py#L45-L81","documentation":"When a provider returns a complete (non-streaming) ModelResponse that LiteLLM must fake as a stream, the generic converter builds a chat.completion.chunk from the response's fields. If any field access or pydantic construction fails (malformed choices, missing attributes on a partial response), the whole conversion is caught and re-raised with the serialized response for debugging.","triggerScenarios":"A provider handler returns a ModelResponse whose choices/usage are malformed or whose types don't fit Choice chunk construction (e.g. custom provider adapter or a mocked response), and stream=True forces the conversion path.","commonSituations":"Writing/testing a custom provider integration that returns hand-built ModelResponse objects; a mocked response object missing expected attributes; version mismatches between litellm core and a provider adapter after upgrade.","solutions":["Inspect the printed model_response in the message — usually a malformed choices list is visible","If building ModelResponse yourself, construct it via ModelResponse(choices=[Choices(delta=..., message=...)]) so fields are well-formed","Call with stream=False to see the raw response error directly instead of the conversion failure","Upgrade litellm — conversion helpers gain field tolerance over versions"],"exampleFix":"# before (custom handler)\nresp = ModelResponse(); resp.choices = [{}]  # malformed\n\n# after\nfrom litellm.types.utils import ModelResponse, Choices, Delta\nresp = ModelResponse(choices=[Choices(index=0, delta=Delta(content='hi'), finish_reason='stop')])","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    for chunk in response:\n        process(chunk)\nexcept ValueError as e:\n    if 'Failed to convert ModelResponse to ModelResponseStream' in str(e):\n        log.error('malformed provider response: %s', e)\n        # fall back to non-streaming call\n    else:\n        raise","preventionTips":["When building custom ModelResponse objects, always use the pydantic constructors (Choices/Delta) rather than assigning raw dicts","Test custom provider adapters with stream=True and stream=False in CI"],"tags":["streaming","internal","custom-provider","type-error"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}