BerriAI/litellm · error · ValueError

model_name not set for LlamaGuard

Error message

model_name not set for LlamaGuard

What it means

ValueError raised in _ENTERPRISE_LlamaGuard.__init__ when neither the constructor's model_name argument nor litellm.llamaguard_model_name is set. LlamaGuard moderation needs a base LLM to classify safety, so the hook refuses to initialize without one.

Source

Thrown at enterprise/litellm_enterprise/enterprise_callbacks/llama_guard.py:34

)  # Adds the parent directory to the system path
import sys
from typing import Literal, Optional

from fastapi import HTTPException

import litellm
from litellm._logging import verbose_proxy_logger
from litellm.integrations.custom_logger import CustomLogger
from litellm.proxy._types import UserAPIKeyAuth
from litellm.types.utils import CallTypesLiteral, Choices, ModelResponse


class _ENTERPRISE_LlamaGuard(CustomLogger):
    # Class variables or attributes
    def __init__(self, model_name: Optional[str] = None):
        _model = model_name or litellm.llamaguard_model_name
        if _model is None:
            raise ValueError("model_name not set for LlamaGuard")
        self.model = _model
        file_path = litellm.llamaguard_unsafe_content_categories
        data = None

        if file_path is not None:
            try:
                with open(file_path, "r") as file:
                    data = file.read()
            except FileNotFoundError:
                raise Exception(f"File not found. file_path={file_path}")
            except Exception as e:
                raise Exception(f"An error occurred: {str(e)}, file_path={file_path}")

        self.unsafe_content_categories = data

        verbose_proxy_logger.debug(
            f"self.unsafe_content_categories: {self.unsafe_content_categories}"
        )

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Set litellm_settings.llamaguard_model_name in the proxy config to a model available in model_list (e.g. an 8B LlamaGuard deployment on Groq, Together, or Ollama).
2. Or pass model_name explicitly when constructing _ENTERPRISE_LlamaGuard in code.
3. Verify the referenced model also exists in model_list so the moderation call itself can route.
4. If LlamaGuard is unwanted, remove the callback from config.

Example fix

# before
litellm_settings:
  callbacks: llama_guard  # no model configured -> ValueError

# after
litellm_settings:
  llamaguard_model_name: groq/llama-guard-3-8b
  callbacks: llama_guard

import litellm

if not (getattr(litellm, "llamaguard_model_name", None) or explicit_model_name):
    raise SystemExit("set litellm_settings.llamaguard_model_name before enabling llama_guard")

from typing import Optional

def has_llamaguard_model(model_name: Optional[str]) -> bool:
    return bool(model_name or getattr(litellm, "llamaguard_model_name", None))

try:
    guard = _ENTERPRISE_LlamaGuard(model_name=name)
except ValueError as e:
    if "model_name not set" in str(e):
        logger.error("configure llamaguard_model_name in litellm_settings")
    raise

When it happens

Trigger: Adding the llama_guard callback/hook without setting llamaguard_model_name in litellm settings (e.g. litellm_settings.llamaguard_model_name: groq/llama-guard-3-8b) and without passing model_name programmatically. Initialization happens at proxy startup or hook registration.

Common situations: Copy-pasting an enterprise config that references the LlamaGuard hook but omitting the model setting; expecting the hook to work self-hosted without any underlying model; name typos in llamaguard_model_name.

Related errors

• File not found. file_path={file_path}
• File not found. blocked_user_list={blocked_user_list}
• An error occurred: {str(e)}, file_path={file_path}
• Missing `LLM_GUARD_API_BASE` from environment
• An error occurred: {str(e)}, blocked_user_list={blocked_user

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/78bd7654183c5213. Report an issue: GitHub.