What causes "Ollama Error - {chunk}" in BerriAI/litellm?

Read the chunk in the message — it contains Ollama's actual error text (e.g. 'model ... not found').

BerriAI/litellm · error · Exception

Ollama Error - {chunk}

Error message

Ollama Error - {chunk}

What it means

In Ollama text-completion streaming, each parsed chunk is checked for an 'error' key. Ollama reports some errors inside an HTTP 200 stream (e.g. model load failure); LiteLLM raises a generic Exception whose message is 'Ollama Error - {chunk}', embedding the raw error payload.

Source

Thrown at litellm/llms/ollama/completion/transformation.py:458

            streaming_response=streaming_response,
            sync_stream=sync_stream,
            json_mode=json_mode,
        )


class OllamaTextCompletionResponseIterator(BaseModelResponseIterator):
    def __init__(self, streaming_response, sync_stream: bool, json_mode: bool | None = False):
        super().__init__(streaming_response, sync_stream, json_mode)
        self.started_reasoning_content: bool = False
        self.finished_reasoning_content: bool = False

    def _handle_string_chunk(self, str_line: str) -> GenericStreamingChunk | ModelResponseStream:
        return self.chunk_parser(json.loads(str_line))

    def chunk_parser(self, chunk: dict) -> GenericStreamingChunk | ModelResponseStream:
        try:
            if "error" in chunk:
                raise Exception(f"Ollama Error - {chunk}")

            text = ""
            is_finished = False
            finish_reason = None
            if chunk["done"] is True:
                text = ""
                is_finished = True
                finish_reason = "stop"
                prompt_eval_count: Final[int | None] = chunk.get("prompt_eval_count", None)
                eval_count: Final[int | None] = chunk.get("eval_count", None)

                usage: ChatCompletionUsageBlock | None = None
                if prompt_eval_count is not None and eval_count is not None:
                    usage = ChatCompletionUsageBlock(
                        prompt_tokens=prompt_eval_count,
                        completion_tokens=eval_count,
                        total_tokens=prompt_eval_count + eval_count,
                    )

View on GitHub (pinned to 6c2dcb801b)

Solutions

Read the chunk in the message — it contains Ollama's actual error text (e.g. 'model ... not found').
Run `ollama list` to confirm the model exists and `ollama run <model>` to verify it loads.
For OOM errors, free memory or use a smaller model/quantization.
Wrap streaming calls in try/except and surface the error instead of assuming stream failure is a network issue.

Defensive patterns

Strategy: try-catch

Try / catch

try:
    for chunk in litellm.text_completion(model="ollama/m", prompt=p, stream=True):
        ...
except Exception as e:
    if str(e).startswith("Ollama Error -"):
        # in-band stream error: message contains the raw Ollama payload
        handle_ollama_error_payload(str(e))

Prevention

Verify models exist with ollama list before streaming
Treat any chunk as potentially fatal — in-band errors arrive after successful chunks
Monitor Ollama server memory to pre-empt OOM mid-stream

When it happens

Trigger: Streaming `litellm.completion(model='ollama/<model>', ...)` (text-completion path) where the model failed to load, does not exist on the server, or ran out of memory mid-stream. The chunk arrives as {"error": "..."} inside the SSE stream and triggers this raise inside chunk_parser.

Common situations: Pulling a model name with a typo or that was never `ollama pull`ed; server OOM while generating; model file corruption; or the model being deleted from the server between requests.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/95dd29b48d546f52. Report an issue: GitHub.