What causes "Missing `LLM_GUARD_API_BASE` from environment" in BerriAI/litellm?

Set LLM_GUARD_API_BASE to your LLM Guard service URL (e.g. http://llm-guard:8000/) in the proxy's environment — include the trailing slash or not, the hook normalizes it.

BerriAI/litellm · error · Exception

Missing `LLM_GUARD_API_BASE` from environment

Error message

Missing `LLM_GUARD_API_BASE` from environment

What it means

Exception raised in the LLM Guard hook's __init__ when the environment variable LLM_GUARD_API_BASE is not set (looked up via get_secret_str). The hook needs a running LLM Guard service (protectai's llm-guard API wrapper) to sanitize prompts, so it refuses to initialize without its base URL.

Source

Thrown at enterprise/litellm_enterprise/enterprise_callbacks/llm_guard.py:37

from litellm.proxy._types import UserAPIKeyAuth
from litellm.secret_managers.main import get_secret_str
from litellm.types.utils import CallTypesLiteral


class _ENTERPRISE_LLMGuard(CustomLogger):
    # Class variables or attributes
    def __init__(
        self,
        mock_testing: bool = False,
        mock_redacted_text: Optional[dict] = None,
    ):
        self.mock_redacted_text = mock_redacted_text
        self.llm_guard_mode = litellm.llm_guard_mode
        if mock_testing is True:  # for testing purposes only
            return
        self.llm_guard_api_base = get_secret_str("LLM_GUARD_API_BASE", None)
        if self.llm_guard_api_base is None:
            raise Exception("Missing `LLM_GUARD_API_BASE` from environment")
        elif not self.llm_guard_api_base.endswith("/"):
            self.llm_guard_api_base += "/"

    def print_verbose(self, print_statement):
        try:
            verbose_proxy_logger.debug(print_statement)
            if litellm.set_verbose:
                print(print_statement)  # noqa
        except Exception:
            pass

    async def moderation_check(self, text: str) -> str:
        """
        Runs the LLM Guard moderation check on ``text``.

        Raises an HTTPException when the content violates the safety policy;
        otherwise returns the sanitized prompt from LLM Guard, falling back to
        the original text when the API does not provide one.

View on GitHub (pinned to 6c2dcb801b)

Solutions

Set LLM_GUARD_API_BASE to your LLM Guard service URL (e.g. http://llm-guard:8000/) in the proxy's environment — include the trailing slash or not, the hook normalizes it.
For Kubernetes, add it to the deployment env/envFrom instead of relying on local shell exports.
Verify with a direct request: curl $LLM_GUARD_API_BASE/analyze/prompt -d '{"prompt":"hi"}'.
If just testing hook wiring, init with mock_testing=True.

Example fix

# before
callbacks: llm_guard  # LLM_GUARD_API_BASE unset -> Exception

# after
export LLM_GUARD_API_BASE=http://llm-guard.internal:8000/
callbacks: llm_guard

Defensive patterns

Strategy: validation

Validate before calling

import os

api_base = os.getenv("LLM_GUARD_API_BASE")
if not api_base:
    raise SystemExit("LLM_GUARD_API_BASE must be set before enabling llm_guard")

Try / catch

try:
    hook = _ENTERPRISE_LLMGuard()
except Exception as e:
    if "LLM_GUARD_API_BASE" in str(e):
        logger.error("export LLM_GUARD_API_BASE=http://llm-guard:8000/ and restart")
    raise

Prevention

Set LLM_GUARD_API_BASE in the deployment spec (K8s env, Compose environment), not just local shell.
Use mock_testing=True in CI to exercise hook wiring without the service.
Add a readiness probe against the LLM Guard service before enabling the hook.
Keep the URL scheme+host stable across environments via config maps.

When it happens

Trigger: Adding the llm_guard hook/callback to the proxy config without exporting LLM_GUARD_API_BASE in the proxy process's environment. mock_testing=True skips the check, so it only fires in real deployments.

Common situations: Enabling LLM Guard integration after reading enterprise docs but forgetting the env var; env var set in a shell but not in the systemd unit/Docker Compose/Kubernetes deployment where the proxy actually runs; secret managers not wired so get_secret_str finds nothing.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/c4b4c96be749a1e6. Report an issue: GitHub.