affaan-m/ECC · warning · RateLimitError

{msg}

Error message

{msg}

What it means

OllamaProvider.generate() raises RateLimitError when the exception message contains '429' or 'rate_limit'. Ollama is local and has no per-key quota, so this typically fires when OLLAMA_NUM_PARALLEL or OLLAMA_MAX_LOADED_MODELS is exceeded (older Ollama defaulted NUM_PARALLEL to 1), or when a reverse proxy/gateway in front of Ollama enforces 429s.

Source

Thrown at src/llm/providers/ollama.py:106

                        id=tc.get("id", ""),
                        name=tc.get("function", {}).get("name", ""),
                        arguments=tc.get("function", {}).get("arguments", {}),
                    )
                    for tc in result["message"]["tool_calls"]
                ]

            return LLMOutput(
                content=content,
                tool_calls=tool_calls,
                model=model,
                stop_reason=result.get("done_reason"),
            )
        except Exception as e:
            msg = str(e)
            if "401" in msg or "connection" in msg.lower():
                raise AuthenticationError(f"Ollama connection failed: {msg}", provider=ProviderType.OLLAMA) from e
            if "429" in msg or "rate_limit" in msg.lower():
                raise RateLimitError(msg, provider=ProviderType.OLLAMA) from e
            if "context" in msg.lower() and "length" in msg.lower():
                raise ContextLengthError(msg, provider=ProviderType.OLLAMA) from e
            raise

    def list_models(self) -> list[ModelInfo]:
        return self._models.copy()

    def validate_config(self) -> bool:
        return bool(self.base_url)

    def get_default_model(self) -> str:
        return self.default_model

View on GitHub (pinned to 01e15490f0)

Solutions

  1. Serialize Ollama calls or raise OLLAMA_NUM_PARALLEL in the server config.
  2. Reduce concurrency of the calling code.
  3. If behind a proxy, raise its rate limit.
  4. Retry with exponential backoff on RateLimitError.

Example fix

// before
output = provider.generate(llm_input)

// after
import time
from llm.core.interface import RateLimitError

for attempt in range(4):
    try:
        output = provider.generate(llm_input)
        break
    except RateLimitError:
        if attempt == 3:
            raise
        time.sleep(2 ** attempt)
Defensive patterns

Strategy: retry

Validate before calling

import os, httpx

def ollama_has_capacity(base_url: str) -> bool:
    # OLLAMA_NUM_PARALLEL defaults vary; this is a liveness check only
    try:
        return httpx.get(f"{base_url.rstrip('/')}/api/tags", timeout=5).status_code == 200
    except Exception:
        return False

Type guard

from llm.core.interface import RateLimitError

def is_ollama_rate_limit(exc: BaseException) -> bool:
    return isinstance(exc, RateLimitError)

Try / catch

import time
from llm.core.interface import RateLimitError

for attempt in range(4):
    try:
        output = provider.generate(llm_input)
        break
    except RateLimitError:
        if attempt == 3:
            raise
        time.sleep(2 ** attempt)

Prevention

When it happens

Trigger: More concurrent requests than OLLAMA_NUM_PARALLEL; a gateway/proxy in front of Ollama returning 429; an Ollama error whose text happens to contain 'rate_limit'.

Common situations: Parallel agent loops hitting a single local Ollama; OLLAMA_NUM_PARALLEL left at default; Ollama routed through nginx with rate limiting; a version change adding rate-limit-style errors for a busy GPU.

Related errors

AI-assisted analysis of affaan-m/ECC@01e15490f0 (2026-08-13). Data as JSON: /api/errors/9abc08756320e104. Report an issue: GitHub.