{"record":{"id":"f6584aa60e470bb8","repo":"BerriAI/litellm","slug":"invalid-content-moderation-response-redacted-tex","errorCode":null,"errorMessage":"Invalid content moderation response: {redacted_text}","messagePattern":"Invalid content moderation response: (.+?)","errorType":"http","errorClass":"HTTPException","httpStatus":500,"severity":"error","filePath":"enterprise/litellm_enterprise/enterprise_callbacks/llm_guard.py","lineNumber":74,"sourceCode":"\n        [TODO] make this more performant for high-throughput scenario\n        \"\"\"\n        try:\n            if self.mock_redacted_text is not None:\n                redacted_text = self.mock_redacted_text\n            else:\n                analyze_url = f\"{self.llm_guard_api_base}analyze/prompt\"\n                verbose_proxy_logger.debug(\"Making request to: %s\", analyze_url)\n                async with aiohttp.ClientSession() as session:\n                    async with session.post(\n                        analyze_url, json={\"prompt\": text}\n                    ) as response:\n                        redacted_text = await response.json()\n            verbose_proxy_logger.debug(\n                f\"LLM Guard: Received response - {redacted_text}\"\n            )\n            if redacted_text is None:\n                raise HTTPException(\n                    status_code=500,\n                    detail={\n                        \"error\": f\"Invalid content moderation response: {redacted_text}\"\n                    },\n                )\n            if redacted_text.get(\"is_valid\", None) is False:\n                raise HTTPException(\n                    status_code=400,\n                    detail={\"error\": \"Violated content safety policy\"},\n                )\n            sanitized_prompt = redacted_text.get(\"sanitized_prompt\")\n            return sanitized_prompt if isinstance(sanitized_prompt, str) else text\n        except Exception as e:\n            verbose_proxy_logger.exception(\n                \"litellm.enterprise.enterprise_hooks.llm_guard::moderation_check - Exception occurred - {}\".format(\n                    str(e)\n                )\n            )","sourceCodeStart":56,"sourceCodeEnd":92,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/enterprise/litellm_enterprise/enterprise_callbacks/llm_guard.py#L56-L92","documentation":"Raised as HTTPException 500 by LLM Guard's moderation_check when the response from the LLM Guard service's /analyze/prompt (or sanitize) endpoint parses to None — i.e. the service returned an empty or non-JSON body that aiohttp's response.json() yielded as null. The hook treats an unparseable moderation response as an internal error rather than passing traffic through.","triggerScenarios":"The LLM Guard service at LLM_GUARD_API_BASE is up enough to accept the POST but returns an empty body, a null JSON, or a non-JSON error page (e.g. a proxy 502 HTML page), making redacted_text None after parsing.","commonSituations":"LLM Guard sidecar crashed mid-request; an ingress/load balancer in front of it returning empty 502/503 bodies; version mismatch where the deployed LLM Guard API no longer returns the expected JSON shape; wrong URL path prefix so a 404 body comes back.","solutions":["Check the LLM Guard service health and logs at LLM_GUARD_API_BASE — it is returning empty/null bodies.","Test directly: curl -X POST $LLM_GUARD_API_BASE/analyze/prompt -H 'Content-Type: application/json' -d '{\"prompt\":\"hello\"}' and confirm valid JSON with is_valid and sanitized_prompt fields.","Fix routing/ingress so /analyze/prompt reaches the real service (no 404/502 interception).","Align LLM Guard API version with what the hook expects."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"import httpx\n\nresp = httpx.post(\n    f\"{LLM_GUARD_API_BASE}analyze/prompt\", json={\"prompt\": \"healthcheck\"}, timeout=5\n)\nassert resp.status_code == 200 and resp.json() is not None, \"LLM Guard returning invalid JSON\"","typeGuard":"from typing import Any, Optional\n\ndef is_valid_guard_response(r: Any) -> bool:\n    return isinstance(r, dict) and isinstance(r.get(\"is_valid\"), bool)","tryCatchPattern":"for attempt in range(3):\n    try:\n        result = await moderation_check(text)\n        break\n    except HTTPException as e:\n        if e.status_code == 500 and \"Invalid content moderation response\" in str(e.detail):\n            await asyncio.sleep(2 ** attempt)  # service hiccup — retry with backoff\n            continue\n        raise","preventionTips":["Monitor the LLM Guard service's health endpoint; alert on non-JSON responses.","Keep the ingress from intercepting /analyze/prompt with 4xx/5xx HTML pages.","Pin compatible versions of the LLM Guard service and the litellm hook.","Circuit-break after repeated 500s rather than retrying indefinitely."],"tags":["enterprise","llm-guard","network","http-500","integration"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}