BerriAI/litellm · error · Exception

Generic Guardrail API failed: {error}

Error message

Generic Guardrail API failed: {error}

What it means

Bare Exception raised in GenericGuardrailAPI._handle_guardrail_request_error when the guardrail call fails and neither fail-open branch applies: i.e. the error is not an unreachable-502/503/504 with unreachable_fallback='fail_open', and fail_on_error is True (its default). It wraps Timeout, HTTPStatusError (any status, including 401/403/404/429), RequestError, and unexpected exceptions from payload building. Note this is a plain Exception, not a dedicated error class.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/generic_guardrail_api/generic_guardrail_api.py:360

        self,
        error: Exception,
        inputs: GenericGuardrailAPIInputs,
        input_type: Literal["request", "response"],
        logging_obj: Optional["LiteLLMLoggingObj"],
        is_unreachable: bool = True,
    ) -> GenericGuardrailAPIInputs:
        unreachable_fail_open: Final = is_unreachable and self.unreachable_fallback == "fail_open"
        if unreachable_fail_open or not self.fail_on_error:
            http_status_code: Final = getattr(getattr(error, "response", None), "status_code", None)
            return self._fail_open_passthrough(
                inputs=inputs,
                input_type=input_type,
                logging_obj=logging_obj,
                error=error,
                **({"http_status_code": http_status_code} if http_status_code else {}),
            )
        verbose_proxy_logger.error("Generic Guardrail API: failed to make request: %s", str(error))
        raise Exception(f"Generic Guardrail API failed: {error}")

    @log_guardrail_information
    async def apply_guardrail(
        self,
        inputs: GenericGuardrailAPIInputs,
        request_data: dict,
        input_type: Literal["request", "response"],
        logging_obj: Optional["LiteLLMLoggingObj"] = None,
    ) -> GenericGuardrailAPIInputs:
        """
        Apply the Generic Guardrail API to the given inputs.

        This is the main method that gets called by the framework.

        Args:
            inputs: Dictionary containing:
                - texts: List of texts to check
                - images: Optional list of images to check

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Check the proxy log 'Generic Guardrail API: failed to make request: <error>' for the underlying cause and status.
2. Curl your service at {api_base}/beta/litellm_basic_guardrail_api with the same x-api-key header to verify reachability and auth.
3. Set fail_on_error: false in litellm_params so guardrail failures pass traffic through (fail-open on any error).
4. For outages only, set unreachable_fallback: fail_open to fail-open specifically on 502/503/504 while still failing on auth errors.
5. Confirm api_base is the service root; the endpoint path is appended automatically.

Example fix

# before
litellm_params:
  guardrail: generic_guardrail_api
  api_base: os.environ/GENERIC_GUARDRAIL_API_BASE
  # fail_on_error defaults to true

# after
litellm_params:
  guardrail: generic_guardrail_api
  api_base: os.environ/GENERIC_GUARDRAIL_API_BASE
  fail_on_error: false

import httpx, os

base = os.getenv('GENERIC_GUARDRAIL_API_BASE')
if base:
    r = httpx.get(f"{base.rstrip('/')}/health", timeout=5)  # adjust to your service's health route
    assert r.status_code < 500, f'guardrail service unhealthy: {r.status_code}'

try:
    result = await guardrail.apply_guardrail(inputs, request_data, input_type, logging_obj)
except Exception as e:  # generic guardrail raises a bare Exception
    if 'Generic Guardrail API failed' in str(e):
        logger.warning('generic guardrail failed, failing open: %s', e)
        result = dict(inputs)
    else:
        raise

When it happens

Trigger: Guardrail service returns 401/403 (bad x-api-key), 404 (wrong path after suffix append), 429, or any 5xx with unreachable_fallback='fail_closed'; connection refused/timeout to api_base; pydantic validation failure while building GenericGuardrailAPIRequest.

Common situations: Custom guardrail service down or scaled to zero; API key header format changed; api_base pointing at the wrong path so the auto-appended /beta/litellm_basic_guardrail_api 404s; fail_on_error left at default true making the guardrail availability gate all LLM traffic.

Related errors

• DeepKeep guardrail API failed: {error}
• {exc}
• Failed to fetch provider mapping: {e}
• api_base is required for Generic Guardrail API. Set GENERIC_
• streaming_sampling_rate must be >= 1 (got {streaming_samplin

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/99eb6d436b4de94d. Report an issue: GitHub.