BerriAI/litellm · error · HTTPException

OpenAI Moderation API request failed

Error message

OpenAI Moderation API request failed

What it means

Runtime HTTPException raised inside OpenAIModerationGuardrail when the POST to {api_base}/moderations returns a non-200 status. The upstream status code is passed through verbatim to the caller, with the response body in detail.details — so a 401 means the guardrail's OpenAI key is bad, 429 means rate-limit/quota, 5xx means OpenAI-side trouble.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/openai/moderations.py:123

        Make a request to the OpenAI Moderation API.
        """
        request_body: Final = {"model": self.model, "input": input_text}

        verbose_proxy_logger.debug("OpenAI Moderation guard request: %s", request_body)

        response: Final = await self.async_handler.post(
            url=f"{self.api_base}/moderations",
            headers={
                "Authorization": f"Bearer {self.api_key}",
                "Content-Type": "application/json",
            },
            json=request_body,
        )

        verbose_proxy_logger.debug("OpenAI Moderation guard response: %s", response.json())

        if response.status_code != 200:
            raise HTTPException(
                status_code=response.status_code,
                detail={
                    "error": "OpenAI Moderation API request failed",
                    "details": response.text,
                },
            )

        from litellm.types.llms.openai import OpenAIModerationResponse

        return OpenAIModerationResponse(**response.json())

    def _check_moderation_result(self, moderation_response: "OpenAIModerationResponse") -> None:
        """
        Check if the moderation response indicates harmful content and raise exception if needed.
        """
        if not moderation_response.results:
            return

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Match the status: 401/403 → fix/rotate the guardrail's api_key; 429 → slow down or reduce guardrail coverage (fewer hooks/models); 5xx → retry later
2. If using a custom api_base, confirm it implements POST /moderations and accepts the same auth
3. Add retry with backoff for 429/5xx since moderation calls are read-only and safe to retry
4. Consider moderating only pre_call (prompts) rather than every chunk if quota is the issue

Example fix

# before
resp = await handler.post(url, ...)  # single shot, 429 kills the request

# after: retry transient upstream statuses
for attempt in range(4):
    resp = await self.async_handler.post(url, headers=..., json=request_body)
    if resp.status_code == 200:
        break
    if resp.status_code in (429, 500, 502, 503) and attempt < 3:
        await asyncio.sleep(float(resp.headers.get("retry-after", 2 ** attempt)))
        continue
    raise HTTPException(status_code=resp.status_code, detail={"error": "OpenAI Moderation API request failed", "details": resp.text})

# Operator-side preflight: verify the guardrail key can actually reach moderations
import httpx

def moderation_key_ok(api_key: str, api_base: str = "https://api.openai.com/v1") -> bool:
    r = httpx.post(
        f"{api_base}/moderations",
        headers={"Authorization": f"Bearer {api_key}"},
        json={"model": "omni-moderation-latest", "input": "ping"},
        timeout=10,
    )
    return r.status_code == 200

for attempt in range(4):
    try:
        r = await client.post(f"{PROXY}/chat/completions", json=payload)
        r.raise_for_status()
        break
    except httpx.HTTPStatusError as e:
        sc = e.response.status_code
        # proxy passes the OpenAI moderation status through verbatim
        if sc in (429, 500, 502, 503) and attempt < 3:
            await asyncio.sleep(float(e.response.headers.get("retry-after", 2 ** attempt)))
            continue
        if sc in (401, 403):
            alert("guardrail OpenAI key invalid")
        raise

When it happens

Trigger: Guardrail api_key revoked or invalid → 401; moderation quota exhausted or org rate-limited → 429; OpenAI incident → 5xx; custom api_base pointing at a gateway that rejects the moderations path (404/400)

Common situations: Rotating the OpenAI key for model calls but forgetting the guardrail config; Azure/other proxies set as api_base without moderations support; bursty pre-call moderation triggering org-wide rate limits

Related errors

• Microsoft Purview DLP: upstream policy evaluation failed
• Lasso API rate limit exceeded
• OpenAI Moderation: guardrail_name is required
• OpenAI Moderation: api_key is required. Set OPENAI_API_KEY e
• Violated OpenAI moderation policy

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/ba45be2d57aed196. Report an issue: GitHub.