BerriAI/litellm · error · OllamaError

KeyError: {e}, Got unexpected response from Ollama: {chunk}

Error message

KeyError: {e}, Got unexpected response from Ollama: {chunk}

What it means

In the Ollama streaming chat path, chunk_parser builds a ModelResponseStream from the Ollama chunk using direct key access (`chunk["model"]` plus message content keys). If any required key is missing, the KeyError is caught and re-raised as an OllamaError (400) that names the missing key and dumps the whole chunk into the message.

Source

Thrown at litellm/llms/ollama/chat/transformation.py:541

                    )
                ]

            usage: Final = ChatCompletionUsageBlock(
                prompt_tokens=chunk.get("prompt_eval_count", 0),
                completion_tokens=chunk.get("eval_count", 0),
                total_tokens=chunk.get("prompt_eval_count", 0) + chunk.get("eval_count", 0),
            )

            return ModelResponseStream(
                id=str(uuid.uuid4()),
                object="chat.completion.chunk",
                created=int(time.time()),  # ollama created_at is in UTC
                usage=usage,
                model=chunk["model"],
                choices=choices,
            )
        except KeyError as e:
            raise OllamaError(
                message=f"KeyError: {e}, Got unexpected response from Ollama: {chunk}",
                status_code=400,
                headers={"Content-Type": "application/json"},
            )
        except Exception as e:
            raise e

View on GitHub (pinned to 6c2dcb801b)

Solutions

Inspect the chunk printed in the error message to see which key is missing and what the payload actually is.
Upgrade (or pin) the Ollama server to a version compatible with your LiteLLM release.
If a proxy sits in front of Ollama, bypass it or configure it to pass SSE frames through unmodified.
Confirm the model you call actually supports chat generation (not an embedding-only model).

Defensive patterns

Strategy: try-catch

Try / catch

try:
    stream = litellm.completion(model="ollama/llama3", messages=msgs, stream=True)
    for chunk in stream:
        ...
except Exception as e:
    if "Got unexpected response from Ollama" in str(e):
        # inspect chunk payload in message; likely version mismatch
        log.error("ollama chunk schema mismatch: %s", e)
        raise

Prevention

Pin Ollama server and LiteLLM versions together in CI
Bypass proxies that rewrite SSE frames
Log the raw chunk from the error message before debugging blindly

When it happens

Trigger: Streaming `litellm.completion(model='ollama/...', stream=True)` where a chunk from the Ollama server lacks the 'model' key or message structure — e.g. an unexpected keep-alive/status line parsed as JSON, a very old Ollama version with a different chunk schema, or a chunk that is an error payload rather than a chat chunk.

Common situations: Ollama server version drift (fields added/removed between releases), hitting a proxy that injects non-chat JSON frames, or Ollama returning an unexpected final chunk shape for non-chat models (embedding-only models called via chat).

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/5cd8663657cbb1ba. Report an issue: GitHub.