BerriAI/litellm · error · HTTPException

Response status code: {response.status_code}

Error message

Response status code: {response.status_code}

What it means

Raised by the Zscaler AI Guard hook's _handle_response when the guardrail API answers with a status code that is neither 200 nor one of the specially handled errors (429, 5xx). Note that _send_request() calls response.raise_for_status() first, so 4xx/5xx actually surface through the generic handler at line 386; this branch therefore fires in practice for 3xx redirects or unexpected 2xx codes (e.g. 201, 202, 204). The proxy aborts the guarded LLM request and returns the upstream status code to the caller.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/zscaler_ai_guard/zscaler_ai_guard.py:358

                else:
                    verbose_proxy_logger.error(
                        "Action field in response is %s, expecting 'ALLOW', 'BLOCK' or 'DETECT'", guardrail_result
                    )
                    user_facing_error = self._create_user_facing_error(
                        f"Action field in response is {guardrail_result}, expecting 'ALLOW', 'BLOCK' or 'DETECT'"
                    )
                    raise HTTPException(status_code=500, detail=user_facing_error)
            else:
                errorMsg: Final = json_response.get("errorMsg", None)
                verbose_proxy_logger.error("statusCode in response: %s, errorMsg: %s", statusCode_in_response, errorMsg)
                user_facing_error = self._create_user_facing_error(
                    f"statusCode in response: {statusCode_in_response}, errorMsg: {errorMsg}"
                )
                raise HTTPException(status_code=500, detail=user_facing_error)
        else:
            verbose_proxy_logger.error("Zscaler AI Guard status_code - %s", response.status_code)
            user_facing_error = self._create_user_facing_error(f"Response status code: {response.status_code}")
            raise HTTPException(status_code=response.status_code, detail=user_facing_error)

    async def make_zscaler_ai_guard_api_call(
        self, zscaler_ai_guard_url, api_key, policy_id, direction, content, **kwargs
    ):
        """
        Makes an API call to the Zscaler AI Guard service and handles retries, errors, and response parsing.
        """

        extra_headers: Final = self._prepare_headers(api_key, **kwargs)

        data: Final = {
            "direction": direction,
            "content": content,
        }
        # Only include policyId when explicitly configured (policy_id >= 1)
        # When policy_id is None, 0, or -1 (default), use resolve-and-execute-policy which infers
        # the policy from headers (e.g., user-api-key-alias)
        if policy_id is not None and policy_id >= 1:

View on GitHub (pinned to 77b7c6c40c)

Solutions

  1. Set zscaler_ai_guard_url to the exact, full AI Guard API endpoint path for your Zscaler cloud (no trailing base URL)
  2. Reproduce the raw call with curl -i from the proxy host using the same headers to see the actual status code and any Location header
  3. Remove or fix any load balancer / corporate proxy in front of the Zscaler URL that may redirect or rewrite responses
  4. Upgrade litellm to the latest release, since Zscaler response handling has changed across versions
  5. If the service genuinely returns an unexpected 2xx for a correct request, open a case with Zscaler support

Example fix

# before (litellm_params in guardrail config)
guardrail: zscaler_ai_guard
zscaler_ai_guard_url: https://api.zscalercloud.net   # base host -> redirect / unexpected status
api_key: os.environ/ZSCALER_API_KEY

# after
api_key: os.environ/ZSCALER_API_KEY
Defensive patterns

Strategy: try-catch

Validate before calling

# Preflight from the proxy host: assert the guardrail endpoint answers 200 without following redirects
import httpx, os

url = os.environ['ZSCALER_AI_GUARD_URL']
resp = httpx.post(
    url,
    headers={'x-api-key': os.environ['ZSCALER_API_KEY']},
    json={'direction': 'REQUEST', 'content': 'ping'},
    follow_redirects=False,
    timeout=10,
)
assert resp.status_code == 200, f'unexpected status {resp.status_code} - fix URL before traffic flows'

Try / catch

from openai import APIStatusError

try:
    resp = client.chat.completions.create(model='gpt-4o', messages=[...])
except APIStatusError as e:
    # Guardrail re-raises the upstream Zscaler status; 3xx here almost always means URL misconfig
    if 300 <= e.status_code < 400 or e.status_code in (201, 202, 204):
        raise RuntimeError('zscaler_ai_guard_url points at a redirect/wrong endpoint') from e
    raise

Prevention

When it happens

Trigger: A /chat/completions call on a proxy with a zscaler_ai_guard guardrail attached while the Zscaler endpoint replies with a redirect (301/302, httpx does not follow redirects by default) or an unusual 2xx such as 201/204. Typical when zscaler_ai_guard_url points at a base path or wrong API version instead of the exact AI Guard endpoint.

Common situations: zscaler_ai_guard_url (or ZSCALER_AI_GUARD_URL env var) missing or having an extra path segment so a different API shape responds; wrong Zscaler cloud/tenant host; an API gateway or corporate proxy in front of Zscaler rewriting status codes; Zscaler API behavior change after an upgrade.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/89e067b25a35ed31. Report an issue: GitHub.