headroomlabs-ai/headroom · warning · HTTPException

Rate limited. Retry after {wait_seconds:.1f}s

Error message

Rate limited. Retry after {wait_seconds:.1f}s

What it means

The Anthropic proxy handler enforces a per-(api-key-prefix, client-IP) rate limit before forwarding upstream. When the limiter's token bucket is exhausted, the handler records a rate_limited metric, releases the pre-upstream semaphore (FastAPI exception handlers skip the handler's finally), and returns HTTP 429 with detail 'Rate limited. Retry after Ns' and a Retry-After header rounded up. The client key is derived from the first 16 chars of the API key plus the resolved client IP (honoring trusted gateway CIDRs only, so header forgery cannot rotate buckets).

Source

Thrown at headroom/proxy/handlers/anthropic.py:1040

                if not api_key:
                    auth = headers.get("authorization", "")
                    if auth.startswith("Bearer "):
                        api_key = auth[7:]
                # Phase F PR-F4: trust ``X-Forwarded-For`` for the rate-limit
                # key only when the connecting peer is in
                # ``HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS``; otherwise we use
                # the direct peer IP and a malicious client cannot rotate
                # rate-limit buckets by forging headers.
                client_ip = resolve_client_ip(request) or "unknown"
                rate_key = f"{api_key[:16]}:{client_ip}" if api_key else client_ip
                allowed, wait_seconds = await self.rate_limiter.check_request(rate_key)
                if not allowed:
                    await self.metrics.record_rate_limited(provider=provider_name)
                    # Unit 4: release the pre-upstream semaphore before we
                    # bail out of the handler via HTTPException — FastAPI's
                    # exception handler will NOT run our ``finally``.
                    await _finalize_pre_upstream()
                    raise HTTPException(
                        status_code=429,
                        detail=f"Rate limited. Retry after {wait_seconds:.1f}s",
                        headers={"Retry-After": str(int(wait_seconds) + 1)},
                    )

            # Budget check
            if self.cost_tracker:
                allowed, remaining = self.cost_tracker.check_budget()
                if not allowed:
                    # Unit 4: release the pre-upstream semaphore before we
                    # bail out of the handler via HTTPException.
                    await _finalize_pre_upstream()
                    raise HTTPException(
                        status_code=429,
                        detail=self.cost_tracker.budget_denial_detail(),
                    )

            # Memory: Get user ID when memory is enabled (fallback to "default" for simple DevEx).

View on GitHub (pinned to 322425c43b)

Solutions

  1. Honor the Retry-After header: sleep that many seconds (+jitter) before the next request.
  2. Raise the limit if the workload legitimately needs it (configure the proxy's rate limiter for this key/IP tier).
  3. Reduce burstiness: batch requests, add client-side concurrency caps or exponential backoff with the server-provided delay.

Example fix

# before
resp = requests.post(url, json=body)  # ignores 429, retries instantly

# after
resp = requests.post(url, json=body)
if resp.status_code == 429:
    time.sleep(int(resp.headers["Retry-After"]) + 1)
    resp = requests.post(url, json=body)
Defensive patterns

Strategy: retry

Validate before calling

# Nothing to validate client-side beyond pacing; check remaining allowance if the proxy exposes a metrics endpoint before sending bursts.

Try / catch

resp = await client.post("/v1/messages", json=body)
if resp.status_code == 429:
    wait = int(resp.headers.get("Retry-After", "5"))
    await asyncio.sleep(wait + random.random())
    resp = await client.post("/v1/messages", json=body)

Prevention

When it happens

Trigger: Exceeding the configured requests-per-window on the /v1/messages (Anthropic-format) endpoint for one api_key+client_ip pair; retry storms from a client that ignores Retry-After; many requests sharing one egress IP.

Common situations: Agentic loops or parallel tool calls bursting above the limiter setting; a shared office NAT IP concentrating many clients into one bucket; a retry loop that treats 429 as a generic failure and immediately re-sends.

Related errors

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/5c93b7ddb03960b4. Report an issue: GitHub.