BerriAI/litellm · error · ValueError

Please set 'CONFIDENT_API_KEY=<>' in your environment variab

Error message

Please set 'CONFIDENT_API_KEY=<>' in your environment variables.

What it means

DeepEvalLogger.__init__ reads CONFIDENT_API_KEY from the environment and raises ValueError if it is missing — you cannot construct the logger without it. Unlike most litellm callbacks, there is no constructor argument for the key; it must be in the process environment before the callback is instantiated.

Source

Thrown at litellm/integrations/deepeval/deepeval.py:30

    TraceSpanApiStatus,
)
from litellm.integrations.deepeval.utils import (
    to_zod_compatible_iso,
    validate_environment,
)


# This file includes the custom callbacks for LiteLLM Proxy
# Once defined, these can be passed in proxy_config.yaml
class DeepEvalLogger(CustomLogger):
    """Logs litellm traces to DeepEval's platform."""

    def __init__(self, *args, **kwargs):
        api_key: Final = os.getenv("CONFIDENT_API_KEY")
        self.litellm_environment = os.getenv("LITELM_ENVIRONMENT", "development")
        validate_environment(self.litellm_environment)
        if not api_key:
            raise ValueError("Please set 'CONFIDENT_API_KEY=<>' in your environment variables.")
        self.api = Api(api_key=api_key)
        super().__init__(*args, **kwargs)

    def log_success_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a success event to DeepEval's platform."""
        self._sync_event_handler(kwargs, response_obj, start_time, end_time, is_success=True)

    def log_failure_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a failure event to DeepEval's platform."""
        self._sync_event_handler(kwargs, response_obj, start_time, end_time, is_success=False)

    async def async_log_failure_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a failure event to DeepEval's platform."""
        await self._async_event_handler(kwargs, response_obj, start_time, end_time, is_success=False)

    async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a success event to DeepEval's platform."""
        await self._async_event_handler(kwargs, response_obj, start_time, end_time, is_success=True)

View on GitHub (pinned to 6c2dcb801b)

Solutions

Export CONFIDENT_API_KEY in the environment of the process that creates DeepEvalLogger (get one from DeepEval's dashboard)
For containers, inject it via docker env / k8s secret so it is present at proxy startup
Verify with a quick check before enabling the callback: python -c "import os; assert os.getenv('CONFIDENT_API_KEY')"

Example fix

# before
litellm.callbacks = ["deepeval"]  # CONFIDENT_API_KEY unset -> ValueError at init

# after
export CONFIDENT_API_KEY=<your key>
litellm.callbacks = ["deepeval"]

Defensive patterns

Strategy: validation

Validate before calling

import os

assert os.getenv("CONFIDENT_API_KEY"), "Set CONFIDENT_API_KEY before enabling the deepeval callback"

Try / catch

try:
    DeepEvalLogger()
except ValueError as e:
    if "CONFIDENT_API_KEY" in str(e):
        raise SystemExit("Missing DeepEval credentials — add CONFIDENT_API_KEY to the environment")
    raise

Prevention

List every callback's required env vars in the deployment checklist and assert them at startup
Inject secrets as process env (k8s secret / docker env), not runtime config files, for this logger

When it happens

Trigger: Adding 'deepeval' to litellm.callbacks or the proxy's callback list without exporting CONFIDENT_API_KEY; also triggered when the callback is dynamically loaded (yaml callbacks) in a subprocess where the var was not propagated.

Common situations: Local dev shell has the var but the proxy runs in Docker/systemd without it; secrets managed only at runtime, not as env vars; CI smoke tests instantiating the logger.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/4380766f4d278413. Report an issue: GitHub.