BerriAI/litellm · error · Exception

DeepEval logging error

Error message

DeepEval logging error: {e.response.text}

What it means

In the vendored DeepEval client, HTTPHandler.post raises httpx.HTTPStatusError via raise_for_status() on any 4xx/5xx from api.confident-ai.com; _http_request converts it to a generic Exception carrying the response body text. Grounded caveat: this vendored copy never returns the httpx response (no `return` statement), so even successful requests later crash send_request with AttributeError on None — a separate latent bug to patch.

Solutions

1. Read the embedded response text — it names the real HTTP status and server reason
2. Validate CONFIDENT_API_KEY on the Confident AI dashboard and re-export it
3. If your account is EU, construct Api(api_key=..., base_url=API_BASE_URL_EU)
4. If you see AttributeError on None instead, patch the vendored client to `return` the response from _http_request

Example fix

# before (vendored api.py)
self.sync_http_handler.post(url=url, headers=headers, json=json, params=params)  # result dropped

# after
return self.sync_http_handler.post(url=url, headers=headers, json=json, params=params)

import httpx
from litellm.integrations.deepeval.api import API_BASE_URL, API_BASE_URL_EU
import os

def deepeval_api_reachable() -> bool:
    base = API_BASE_URL_EU if os.getenv("DEEPEVAL_REGION", "us") == "eu" else API_BASE_URL
    try:
        r = httpx.post(f"{base}/v1/tracing", headers={"CONFIDENT_API_KEY": os.getenv("CONFIDENT_API_KEY", "")}, json={})
        return r.status_code < 500  # auth errors still prove reachability
    except httpx.HTTPError:
        return False

try:
    api.send_request(HttpMethods.POST, Endpoints.TRACING_ENDPOINT, body=body)
except Exception as e:  # generic Exception by design; match on text
    if "logging error" in str(e):
        litellm.verbose_logger.exception("deepeval telemetry failed (non-2xx): %s", e)
    else:
        raise

When it happens

Trigger: Invalid or expired CONFIDENT_API_KEY (401/403); EU account hitting the default non-EU API_BASE_URL (403/404); tracing payload rejected as malformed (400); Confident AI outage (5xx).

Common situations: Key rotated or revoked on the Confident AI dashboard; LITELLM_ENVIRONMENT or endpoint body shaped wrong for the account plan; proxy servers adding unexpected status codes.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• res.json().get("error", res.text)
• Failed to fetch models from Lemonade. Status code
• JWT Auth: OIDC discovery endpoint
• OIDC UserInfo endpoint returned status
• Only POST requests are supported

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/f0469d07329f450b. Report an issue: GitHub.

Appendix: source

Thrown at litellm/integrations/deepeval/api.py:77

            "CONFIDENT_API_KEY": api_key,
        }
        # using the global non-eu variable for base url
        self.base_api_url = base_url or API_BASE_URL
        self.sync_http_handler = HTTPHandler()
        self.async_http_handler = get_async_httpx_client(llm_provider=httpxSpecialProvider.LoggingCallback)

    def _http_request(self, method: str, url: str, headers=None, json=None, params=None):
        if method != "POST":
            raise Exception("Only POST requests are supported")
        try:
            self.sync_http_handler.post(
                url=url,
                headers=headers,
                json=json,
                params=params,
            )
        except httpx.HTTPStatusError as e:
            raise Exception(f"DeepEval logging error: {e.response.text}")
        except Exception as e:
            raise e

    def send_request(self, method: HttpMethods, endpoint: Endpoints, body=None, params=None):
        url: Final = f"{self.base_api_url}{endpoint.value}"
        res: Final = self._http_request(
            method=method.value,
            url=url,
            headers=self._headers,
            json=body,
            params=params,
        )

        if res.status_code == 200:
            try:
                return res.json()
            except ValueError:
                return res.text

View on GitHub (pinned to 77b7c6c40c)