microsoft/semantic-kernel · error · ServiceInvalidResponseError

Invalid response type from Ollama streaming chat completion.

Error message

Invalid response type from Ollama streaming chat completion. Expected AsyncIterator but got {type(response_object)}.

What it means

Raised as ServiceInvalidResponseError when the streaming call `self.client.chat(..., stream=True)` returns something that is NOT an `AsyncIterator`. The streaming path must iterate parts asynchronously; a non-iterable result means the client/server did not enter streaming mode as expected.

Source

Thrown at python/semantic_kernel/connectors/ai/ollama/services/ollama_chat_completion.py:199

        chat_history: "ChatHistory",
        settings: "PromptExecutionSettings",
        function_invoke_attempt: int = 0,
    ) -> AsyncGenerator[list["StreamingChatMessageContent"], Any]:
        if not isinstance(settings, OllamaChatPromptExecutionSettings):
            settings = self.get_prompt_execution_settings_from_settings(settings)
        assert isinstance(settings, OllamaChatPromptExecutionSettings)  # nosec

        prepared_chat_history = self._prepare_chat_history_for_request(chat_history)

        response_object = await self.client.chat(
            model=self.ai_model_id,
            messages=prepared_chat_history,
            stream=True,
            **settings.prepare_settings_dict(),
        )

        if not isinstance(response_object, AsyncIterator):
            raise ServiceInvalidResponseError(
                "Invalid response type from Ollama streaming chat completion. "
                f"Expected AsyncIterator but got {type(response_object)}."
            )

        async for part in response_object:
            if isinstance(part, ChatResponse):
                yield [self._create_streaming_chat_message_content_from_chat_response(part, function_invoke_attempt)]
                continue
            if isinstance(part, Mapping):
                yield [self._create_streaming_chat_message_content(part, function_invoke_attempt)]
                continue
            raise ServiceInvalidResponseError(
                "Invalid response type from Ollama streaming chat completion. "
                f"Expected mapping or ChatResponse but got {type(part)}."
            )

    # endregion

View on GitHub (pinned to c028a0c7dc)

Solutions

1. If using a custom client, make `.chat(..., stream=True)` return an AsyncIterator yielding parts.
2. Pin/align the ollama python SDK version with the connector's expectations.
3. If the server cannot stream, call the non-streaming path (`stream=False`) instead of the streaming kernel method.
4. In tests, return an async generator: `async def chat(...): yield ChatResponse(...)`.

Example fix

# before (fake client returns a plain dict)
async def chat(self, **kw):
    return {'message': {...}}

# after - streaming returns an async iterator
async def chat(self, **kw):
    assert kw.get('stream')
    yield ChatResponse.model_validate({'message': {'role':'assistant','content':'hi'}})

# Smoke-test streaming returns an async iterator
resp = await svc.client.chat(model=svc.ai_model_id, messages=[{'role':'user','content':'hi'}], stream=True)
from collections.abc import AsyncIterator
assert isinstance(resp, AsyncIterator), f'stream=True must yield AsyncIterator, got {type(resp)}'

from collections.abc import AsyncIterator

def is_ollama_stream_iter(obj) -> bool:
    return isinstance(obj, AsyncIterator)

from semantic_kernel.exceptions import ServiceInvalidResponseError
try:
    async for chunk in svc._inner_get_streaming_chat_message_contents(chat_history, settings):
        ...
except ServiceInvalidResponseError as e:
    if 'Expected AsyncIterator' in str(e):
        raise RuntimeError('client did not stream; use non-streaming path or fix client') from e
    raise

When it happens

Trigger: The ollama client returns a single ChatResponse (non-streaming) despite stream=True - e.g. a custom client ignoring the stream flag, an SDK version that changed streaming behavior, or a server/proxy that degrades streaming into a single response.

Common situations: A mock client whose `.chat()` returns a coroutine resolving to a dict instead of an async iterator; an ollama reverse proxy that buffers and returns one object; SDK version mismatch changing the streaming return type.

Related errors

• Invalid response type from Ollama streaming chat completion.
• Invalid response type from Ollama chat completion. Expected
• No message content found in response part.
• No message content found in response.
• The _inner_get_streaming_chat_message_contents method is not

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/d2c8560415551443. Report an issue: GitHub.