BerriAI/litellm · error · CiscoAIDefenseGuardrailAPIError

Cisco AI Defense {surface} API call timed out after {self.ti

Error message

Cisco AI Defense {surface} API call timed out after {self.timeout}s

What it means

CiscoAIDefenseGuardrailAPIError raised when the httpx request to the Cisco AI Defense API raises httpx.TimeoutException — the inspection call exceeded the configured timeout (self.timeout seconds; Cisco default or the value passed in the guardrail config). The original timeout exception is chained (__cause__) so you can see connect vs read timeout details in the traceback.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense.py:834

                timeout=self.timeout,
            )
            response: Final = await self.async_handler.client.send(
                request,
                follow_redirects=False,
            )
            response.raise_for_status()
        except httpx.HTTPStatusError as exc:
            status_code: Final = exc.response.status_code if exc.response is not None else 0
            body_snippet = ""
            try:
                body_snippet = exc.response.text[:500] if exc.response else ""
            except Exception:
                body_snippet = ""
            raise CiscoAIDefenseGuardrailAPIError(
                f"Cisco AI Defense {surface} API returned HTTP {status_code}: {body_snippet}"
            ) from exc
        except httpx.TimeoutException as exc:
            raise CiscoAIDefenseGuardrailAPIError(
                f"Cisco AI Defense {surface} API call timed out after {self.timeout}s"
            ) from exc
        except httpx.RequestError as exc:
            raise CiscoAIDefenseGuardrailAPIError(f"Cisco AI Defense {surface} API request failed: {exc}") from exc

        try:
            return response.json()
        except ValueError as exc:
            raise CiscoAIDefenseGuardrailAPIError(
                f"Cisco AI Defense {surface} API returned a non-JSON response"
            ) from exc

    def _build_headers(self) -> dict[str, str]:
        return {
            CISCO_API_KEY_HEADER: self.api_key,
            "Content-Type": "application/json",
            "Accept": "application/json",
            "User-Agent": f"litellm/{litellm_version}",

View on GitHub (pinned to 77b7c6c40c)

Solutions

Increase the timeout: pass timeout: <seconds> in the guardrail's litellm_params (it is plumbed through to httpx).
Set fallback_on_error: allow so timeouts let the request proceed unscanned instead of failing the call.
Check Cisco status/network path (proxy egress, DNS) if timeouts are new or environment-wide.
If Cisco-side latency is chronic, scan only critical hooks (e.g., drop post_call) to halve inspection calls per request.

Example fix

# before — default timeout, timeouts fail the call
litellm_params:
  guardrail: cisco_ai_defense

# after
litellm_params:
  guardrail: cisco_ai_defense
  timeout: 10
  fallback_on_error: allow

Defensive patterns

Strategy: retry

Validate before calling

# Smoke-test inspection latency before enabling on all traffic
import httpx, os, time, statistics
samples = []
for _ in range(10):
    t0 = time.monotonic()
    try:
        httpx.get(os.environ["CISCO_AI_DEFENSE_API_BASE"], timeout=10)
    except Exception: pass
    samples.append(time.monotonic() - t0)
p99 = sorted(samples)[-1]
assert p99 < configured_timeout, f"timeout={configured_timeout}s < observed p99={p99:.2f}s — raise guardrail timeout"

Try / catch

retryable = None
try:
    resp = litellm.completion(...)
except Exception as e:
    if isinstance(getattr(e, "__cause__", None), httpx.TimeoutException) and "timed out after" in str(e):
        retryable = e
if retryable is not None and attempt < 2:
    time.sleep(1.5 ** attempt)  # then retry; persistent timeouts -> check Cisco status

Prevention

Set guardrail timeout to ≥3x observed p99 inspection latency.
Enable fallback_on_error: allow so a slow scanner degrades to unscanned instead of failing.
Hook fewer event surfaces (e.g., pre_call only) to halve per-request inspection time.
Track timeout rate as a KPI; sustained growth usually precedes a Cisco incident.

When it happens

Trigger: Every inspection surface (request scan pre-call, response scan post-call) goes through the same send(); slow Cisco responses under load, cross-region latency to the API base, an aggressive timeout setting (e.g., 1-2s), or DNS/connect stalls past the threshold produce this error.

Common situations: Default timeout too tight for p99 Cisco latency during incidents; streaming endpoints adding response-side scanning latency; privateCisco deployments behind slow corporate proxies; retries absent so a single slow call surfaces to the user request.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/11a75867613b182e. Report an issue: GitHub.