BerriAI/litellm · error · ValueError

Please set 'CONFIDENT_API_KEY=<>' in your environment…

Error message

Please set 'CONFIDENT_API_KEY=<>' in your environment variables.

What it means

DeepEvalLogger.__init__ reads CONFIDENT_API_KEY from the environment and raises ValueError when absent — before any super().__init__. Because litellm instantiates custom loggers when loading callbacks (proxy config.yaml litellm_settings.callbacks, or callbacks= at import), a missing key fails logger construction and typically aborts proxy startup or callback registration.

Solutions

Export CONFIDENT_API_KEY (exact name) in the environment of the process that builds the logger and verify with printenv
For docker/k8s, inject it via env: / Secret and restart
If DeepEval logging is optional, gate the callback on the key's presence in config so startup never hard-fails

Example fix

# before (config.yaml)
litellm_settings:
  callbacks: ["deepeval"]   # startup ValueError without key

# after
export CONFIDENT_API_KEY=<key>
# or gate it programmatically:
import os, litellm
if os.getenv("CONFIDENT_API_KEY"):
    from litellm.integrations.deepeval import DeepEvalLogger
    litellm.callbacks = [DeepEvalLogger()]

Defensive patterns

Strategy: validation

Validate before calling

import os

def deepeval_configured() -> bool:
    return bool(os.getenv("CONFIDENT_API_KEY"))

# in startup:
# if deepeval_configured(): register DeepEvalLogger() else skip (or warn)

Try / catch

try:
    from litellm.integrations.deepeval import DeepEvalLogger
    deepeval_logger = DeepEvalLogger()
except ValueError as e:
    if "CONFIDENT_API_KEY" in str(e):
        deepeval_logger = None  # degrade to no telemetry, keep service up
    else:
        raise

Prevention

Gate optional telemetry callbacks on their env var at config time
Inject secrets via the platform (k8s Secret, docker env) and verify with printenv in a smoke test
Name the variable exactly CONFIDENT_API_KEY — not DEEPEVAL_API_KEY

When it happens

Trigger: litellm_settings: callbacks: ["deepeval"] in proxy config without CONFIDENT_API_KEY; direct DeepEvalLogger() instantiation in tests; CI/containers where the secret was not injected; key named differently (DEEPEVAL_API_KEY).

Common situations: Local works (key in shell) but deployed container lacks the env; .env file not loaded; secret stored under the official deepeval variable name rather than CONFIDENT_API_KEY.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/4380766f4d278413. Report an issue: GitHub.

Appendix: source

Thrown at litellm/integrations/deepeval/deepeval.py:30

    TraceSpanApiStatus,
)
from litellm.integrations.deepeval.utils import (
    to_zod_compatible_iso,
    validate_environment,
)


# This file includes the custom callbacks for LiteLLM Proxy
# Once defined, these can be passed in proxy_config.yaml
class DeepEvalLogger(CustomLogger):
    """Logs litellm traces to DeepEval's platform."""

    def __init__(self, *args, **kwargs):
        api_key: Final = os.getenv("CONFIDENT_API_KEY")
        self.litellm_environment = os.getenv("LITELM_ENVIRONMENT", "development")
        validate_environment(self.litellm_environment)
        if not api_key:
            raise ValueError("Please set 'CONFIDENT_API_KEY=<>' in your environment variables.")
        self.api = Api(api_key=api_key)
        super().__init__(*args, **kwargs)

    def log_success_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a success event to DeepEval's platform."""
        self._sync_event_handler(kwargs, response_obj, start_time, end_time, is_success=True)

    def log_failure_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a failure event to DeepEval's platform."""
        self._sync_event_handler(kwargs, response_obj, start_time, end_time, is_success=False)

    async def async_log_failure_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a failure event to DeepEval's platform."""
        await self._async_event_handler(kwargs, response_obj, start_time, end_time, is_success=False)

    async def async_log_success_event(self, kwargs, response_obj, start_time, end_time):
        """Logs a success event to DeepEval's platform."""
        await self._async_event_handler(kwargs, response_obj, start_time, end_time, is_success=True)

View on GitHub (pinned to 77b7c6c40c)