BerriAI/litellm · error · HTTPException

Invalid content moderation response: {redacted_text}

Error message

Invalid content moderation response: {redacted_text}

What it means

Raised as HTTPException 500 by LLM Guard's moderation_check when the response from the LLM Guard service's /analyze/prompt (or sanitize) endpoint parses to None — i.e. the service returned an empty or non-JSON body that aiohttp's response.json() yielded as null. The hook treats an unparseable moderation response as an internal error rather than passing traffic through.

Source

Thrown at enterprise/litellm_enterprise/enterprise_callbacks/llm_guard.py:74

        [TODO] make this more performant for high-throughput scenario
        """
        try:
            if self.mock_redacted_text is not None:
                redacted_text = self.mock_redacted_text
            else:
                analyze_url = f"{self.llm_guard_api_base}analyze/prompt"
                verbose_proxy_logger.debug("Making request to: %s", analyze_url)
                async with aiohttp.ClientSession() as session:
                    async with session.post(
                        analyze_url, json={"prompt": text}
                    ) as response:
                        redacted_text = await response.json()
            verbose_proxy_logger.debug(
                f"LLM Guard: Received response - {redacted_text}"
            )
            if redacted_text is None:
                raise HTTPException(
                    status_code=500,
                    detail={
                        "error": f"Invalid content moderation response: {redacted_text}"
                    },
                )
            if redacted_text.get("is_valid", None) is False:
                raise HTTPException(
                    status_code=400,
                    detail={"error": "Violated content safety policy"},
                )
            sanitized_prompt = redacted_text.get("sanitized_prompt")
            return sanitized_prompt if isinstance(sanitized_prompt, str) else text
        except Exception as e:
            verbose_proxy_logger.exception(
                "litellm.enterprise.enterprise_hooks.llm_guard::moderation_check - Exception occurred - {}".format(
                    str(e)
                )
            )

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Check the LLM Guard service health and logs at LLM_GUARD_API_BASE — it is returning empty/null bodies.
2. Test directly: curl -X POST $LLM_GUARD_API_BASE/analyze/prompt -H 'Content-Type: application/json' -d '{"prompt":"hello"}' and confirm valid JSON with is_valid and sanitized_prompt fields.
3. Fix routing/ingress so /analyze/prompt reaches the real service (no 404/502 interception).
4. Align LLM Guard API version with what the hook expects.

import httpx

resp = httpx.post(
    f"{LLM_GUARD_API_BASE}analyze/prompt", json={"prompt": "healthcheck"}, timeout=5
)
assert resp.status_code == 200 and resp.json() is not None, "LLM Guard returning invalid JSON"

from typing import Any, Optional

def is_valid_guard_response(r: Any) -> bool:
    return isinstance(r, dict) and isinstance(r.get("is_valid"), bool)

for attempt in range(3):
    try:
        result = await moderation_check(text)
        break
    except HTTPException as e:
        if e.status_code == 500 and "Invalid content moderation response" in str(e.detail):
            await asyncio.sleep(2 ** attempt)  # service hiccup — retry with backoff
            continue
        raise

When it happens

Trigger: The LLM Guard service at LLM_GUARD_API_BASE is up enough to accept the POST but returns an empty body, a null JSON, or a non-JSON error page (e.g. a proxy 502 HTML page), making redacted_text None after parsing.

Common situations: LLM Guard sidecar crashed mid-request; an ingress/load balancer in front of it returning empty 502/503 bodies; version mismatch where the deployed LLM Guard API no longer returns the expected JSON shape; wrong URL path prefix so a 404 body comes back.

Related errors

• Internal Server Error
• Missing `LLM_GUARD_API_BASE` from environment
• Violated content safety policy
• Error saving email settings to general_settings: {str(e)}
• Keyword banned. Keyword={word}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/f6584aa60e470bb8. Report an issue: GitHub.