{"record":{"id":"5c93b7ddb03960b4","repo":"headroomlabs-ai/headroom","slug":"rate-limited-retry-after-wait-seconds-1f-s","errorCode":null,"errorMessage":"Rate limited. Retry after {wait_seconds:.1f}s","messagePattern":"Rate limited\\. Retry after (.+?)s","errorType":"http","errorClass":"HTTPException","httpStatus":429,"severity":"warning","filePath":"headroom/proxy/handlers/anthropic.py","lineNumber":1040,"sourceCode":"                if not api_key:\n                    auth = headers.get(\"authorization\", \"\")\n                    if auth.startswith(\"Bearer \"):\n                        api_key = auth[7:]\n                # Phase F PR-F4: trust ``X-Forwarded-For`` for the rate-limit\n                # key only when the connecting peer is in\n                # ``HEADROOM_PROXY_TRUSTED_GATEWAY_CIDRS``; otherwise we use\n                # the direct peer IP and a malicious client cannot rotate\n                # rate-limit buckets by forging headers.\n                client_ip = resolve_client_ip(request) or \"unknown\"\n                rate_key = f\"{api_key[:16]}:{client_ip}\" if api_key else client_ip\n                allowed, wait_seconds = await self.rate_limiter.check_request(rate_key)\n                if not allowed:\n                    await self.metrics.record_rate_limited(provider=provider_name)\n                    # Unit 4: release the pre-upstream semaphore before we\n                    # bail out of the handler via HTTPException — FastAPI's\n                    # exception handler will NOT run our ``finally``.\n                    await _finalize_pre_upstream()\n                    raise HTTPException(\n                        status_code=429,\n                        detail=f\"Rate limited. Retry after {wait_seconds:.1f}s\",\n                        headers={\"Retry-After\": str(int(wait_seconds) + 1)},\n                    )\n\n            # Budget check\n            if self.cost_tracker:\n                allowed, remaining = self.cost_tracker.check_budget()\n                if not allowed:\n                    # Unit 4: release the pre-upstream semaphore before we\n                    # bail out of the handler via HTTPException.\n                    await _finalize_pre_upstream()\n                    raise HTTPException(\n                        status_code=429,\n                        detail=self.cost_tracker.budget_denial_detail(),\n                    )\n\n            # Memory: Get user ID when memory is enabled (fallback to \"default\" for simple DevEx).","sourceCodeStart":1022,"sourceCodeEnd":1058,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/proxy/handlers/anthropic.py#L1022-L1058","documentation":"The Anthropic proxy handler enforces a per-(api-key-prefix, client-IP) rate limit before forwarding upstream. When the limiter's token bucket is exhausted, the handler records a rate_limited metric, releases the pre-upstream semaphore (FastAPI exception handlers skip the handler's finally), and returns HTTP 429 with detail 'Rate limited. Retry after Ns' and a Retry-After header rounded up. The client key is derived from the first 16 chars of the API key plus the resolved client IP (honoring trusted gateway CIDRs only, so header forgery cannot rotate buckets).","triggerScenarios":"Exceeding the configured requests-per-window on the /v1/messages (Anthropic-format) endpoint for one api_key+client_ip pair; retry storms from a client that ignores Retry-After; many requests sharing one egress IP.","commonSituations":"Agentic loops or parallel tool calls bursting above the limiter setting; a shared office NAT IP concentrating many clients into one bucket; a retry loop that treats 429 as a generic failure and immediately re-sends.","solutions":["Honor the Retry-After header: sleep that many seconds (+jitter) before the next request.","Raise the limit if the workload legitimately needs it (configure the proxy's rate limiter for this key/IP tier).","Reduce burstiness: batch requests, add client-side concurrency caps or exponential backoff with the server-provided delay."],"exampleFix":"# before\nresp = requests.post(url, json=body)  # ignores 429, retries instantly\n\n# after\nresp = requests.post(url, json=body)\nif resp.status_code == 429:\n    time.sleep(int(resp.headers[\"Retry-After\"]) + 1)\n    resp = requests.post(url, json=body)","handlingStrategy":"retry","validationCode":"# Nothing to validate client-side beyond pacing; check remaining allowance if the proxy exposes a metrics endpoint before sending bursts.","typeGuard":null,"tryCatchPattern":"resp = await client.post(\"/v1/messages\", json=body)\nif resp.status_code == 429:\n    wait = int(resp.headers.get(\"Retry-After\", \"5\"))\n    await asyncio.sleep(wait + random.random())\n    resp = await client.post(\"/v1/messages\", json=body)","preventionTips":["Always read the Retry-After header on 429 and sleep at least that long with jitter.","Cap client concurrency below the proxy's configured limit per key/IP.","Instrument rate_limited metrics so bursts are visible before users report them."],"tags":["rate-limit","http-429","proxy","retry-after","throttling"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}