{"record":{"id":"11a75867613b182e","repo":"BerriAI/litellm","slug":"cisco-ai-defense-surface-api-call-timed-out-afte","errorCode":null,"errorMessage":"Cisco AI Defense {surface} API call timed out after {self.timeout}s","messagePattern":"Cisco AI Defense (.+?) API call timed out after (.+?)s","errorType":"exception","errorClass":"CiscoAIDefenseGuardrailAPIError","httpStatus":null,"severity":"error","filePath":"litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense.py","lineNumber":834,"sourceCode":"                timeout=self.timeout,\n            )\n            response: Final = await self.async_handler.client.send(\n                request,\n                follow_redirects=False,\n            )\n            response.raise_for_status()\n        except httpx.HTTPStatusError as exc:\n            status_code: Final = exc.response.status_code if exc.response is not None else 0\n            body_snippet = \"\"\n            try:\n                body_snippet = exc.response.text[:500] if exc.response else \"\"\n            except Exception:\n                body_snippet = \"\"\n            raise CiscoAIDefenseGuardrailAPIError(\n                f\"Cisco AI Defense {surface} API returned HTTP {status_code}: {body_snippet}\"\n            ) from exc\n        except httpx.TimeoutException as exc:\n            raise CiscoAIDefenseGuardrailAPIError(\n                f\"Cisco AI Defense {surface} API call timed out after {self.timeout}s\"\n            ) from exc\n        except httpx.RequestError as exc:\n            raise CiscoAIDefenseGuardrailAPIError(f\"Cisco AI Defense {surface} API request failed: {exc}\") from exc\n\n        try:\n            return response.json()\n        except ValueError as exc:\n            raise CiscoAIDefenseGuardrailAPIError(\n                f\"Cisco AI Defense {surface} API returned a non-JSON response\"\n            ) from exc\n\n    def _build_headers(self) -> dict[str, str]:\n        return {\n            CISCO_API_KEY_HEADER: self.api_key,\n            \"Content-Type\": \"application/json\",\n            \"Accept\": \"application/json\",\n            \"User-Agent\": f\"litellm/{litellm_version}\",","sourceCodeStart":816,"sourceCodeEnd":852,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/proxy/guardrails/guardrail_hooks/cisco_ai_defense/cisco_ai_defense.py#L816-L852","documentation":"CiscoAIDefenseGuardrailAPIError raised when the httpx request to the Cisco AI Defense API raises httpx.TimeoutException — the inspection call exceeded the configured timeout (self.timeout seconds; Cisco default or the value passed in the guardrail config). The original timeout exception is chained (__cause__) so you can see connect vs read timeout details in the traceback.","triggerScenarios":"Every inspection surface (request scan pre-call, response scan post-call) goes through the same send(); slow Cisco responses under load, cross-region latency to the API base, an aggressive timeout setting (e.g., 1-2s), or DNS/connect stalls past the threshold produce this error.","commonSituations":"Default timeout too tight for p99 Cisco latency during incidents; streaming endpoints adding response-side scanning latency; privateCisco deployments behind slow corporate proxies; retries absent so a single slow call surfaces to the user request.","solutions":["Increase the timeout: pass timeout: <seconds> in the guardrail's litellm_params (it is plumbed through to httpx).","Set fallback_on_error: allow so timeouts let the request proceed unscanned instead of failing the call.","Check Cisco status/network path (proxy egress, DNS) if timeouts are new or environment-wide.","If Cisco-side latency is chronic, scan only critical hooks (e.g., drop post_call) to halve inspection calls per request."],"exampleFix":"# before — default timeout, timeouts fail the call\nlitellm_params:\n  guardrail: cisco_ai_defense\n\n# after\nlitellm_params:\n  guardrail: cisco_ai_defense\n  timeout: 10\n  fallback_on_error: allow","handlingStrategy":"retry","validationCode":"# Smoke-test inspection latency before enabling on all traffic\nimport httpx, os, time, statistics\nsamples = []\nfor _ in range(10):\n    t0 = time.monotonic()\n    try:\n        httpx.get(os.environ[\"CISCO_AI_DEFENSE_API_BASE\"], timeout=10)\n    except Exception: pass\n    samples.append(time.monotonic() - t0)\np99 = sorted(samples)[-1]\nassert p99 < configured_timeout, f\"timeout={configured_timeout}s < observed p99={p99:.2f}s — raise guardrail timeout\"","typeGuard":null,"tryCatchPattern":"retryable = None\ntry:\n    resp = litellm.completion(...)\nexcept Exception as e:\n    if isinstance(getattr(e, \"__cause__\", None), httpx.TimeoutException) and \"timed out after\" in str(e):\n        retryable = e\nif retryable is not None and attempt < 2:\n    time.sleep(1.5 ** attempt)  # then retry; persistent timeouts -> check Cisco status","preventionTips":["Set guardrail timeout to ≥3x observed p99 inspection latency.","Enable fallback_on_error: allow so a slow scanner degrades to unscanned instead of failing.","Hook fewer event surfaces (e.g., pre_call only) to halve per-request inspection time.","Track timeout rate as a KPI; sustained growth usually precedes a Cisco incident."],"tags":["litellm","guardrails","cisco-ai-defense","timeout","network"],"backgroundTag":"request-timeout","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T23:17:16.201Z"}