BerriAI/litellm · warning · Exception

Response from datadog API status_code

Error message

Response from datadog API status_code: {response.status_code}, text: {response.text}

What it means

DatadogLogger's synchronous event logging POSTs the logging payload to the Datadog logs intake endpoint and expects HTTP 202 Accepted. response.raise_for_status() runs first, so true 4xx/5xx replies surface as httpx.HTTPStatusError; this generic Exception fires only when the server answers 2xx but not 202. The whole handler is wrapped in `except Exception` and logged via verbose_logger.exception, so it never propagates to the caller — the log event is simply lost.

Solutions

Enable verbose logging (litellm.set_verbose=True or LITELLM_LOG=DEBUG) and read the logged status_code and response text for the exact rejection reason
Check for a DD_BASE_URL override and either unset it or point it at the correct logs intake for your DD_SITE (https://http-intake.logs.<DD_SITE>)
Verify the API key by curl-ing the intake endpoint directly with a tiny payload and confirming a 202
If a proxy sits in front, confirm it forwards the request and passes Datadog's 202 back untouched

Example fix

# before
export DD_BASE_URL=https://api.us5.datadoghq.com   # wrong: generic API host answers non-202

# after
unset DD_BASE_URL                                    # use default intake for DD_SITE
export DD_SITE=us5.datadoghq.com
export DD_API_KEY=<key>
curl -s -o /dev/null -w '%{http_code}' -X POST "https://http-intake.logs.us5.datadoghq.com/api/v2/logs" -H "DD-API-KEY: $DD_API_KEY" -H 'Content-Type: application/json' -d '[{"message":"ping"}]'  # expect 202

Defensive patterns

Strategy: validation

Validate before calling

import os, httpx

def datadog_intake_reachable() -> bool:
    """Pre-flight: logs intake must answer 202 to a tiny POST before enabling the callback."""
    site = os.getenv("DD_SITE", "datadoghq.com")
    key = os.getenv("DD_API_KEY")
    if not key:
        return False
    base = os.getenv("DD_BASE_URL") or f"https://http-intake.logs.{site}"
    r = httpx.post(f"{base}/api/v2/logs", headers={"DD-API-KEY": key}, json=[{"message": "preflight"}])
    return r.status_code == 202

Try / catch

// Raised inside litellm's callback and already logged; no caller-side catch applies.
// Monitor verbose logs for 'Datadog Layer Error' instead of catching.

Prevention

Run the intake pre-flight check in CI before deploying with the datadog callback
Never point DD_BASE_URL at a generic API host; verify the URL answers 202 to a test POST
Alert on repeated 'Datadog Layer Error' log lines to catch silent telemetry loss

When it happens

Trigger: Datadog callback enabled in direct-API mode and the intake URL does not point at the real logs intake (e.g. DD_BASE_URL override pointing at a generic Datadog API endpoint that answers 200), an intermediate proxy/gateway that returns its own 2xx, or an agent forwarder that replies with a non-202 success code.

Common situations: Setting DD_BASE_URL to a wrong host while also setting DD_API_KEY (so the env checks pass), routing traffic through a corporate proxy that swallows the response, or a Datadog-side API change. Bad API keys usually manifest as HTTPStatusError from raise_for_status instead of this message.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/ab7429b91d75cbcf. Report an issue: GitHub.

Appendix: source

Thrown at litellm/integrations/datadog/datadog.py:509

                    start_time=start_time,
                    end_time=end_time,
                )

            # Build headers
            headers: Final = {}
            # Add API key if available (required for direct API, optional for agent)
            if self.DD_API_KEY:
                headers["DD-API-KEY"] = self.DD_API_KEY

            response: Final = self.sync_client.post(
                url=self.intake_url,
                json=dd_payload,
                headers=headers,
            )

            response.raise_for_status()
            if response.status_code != 202:
                raise Exception(f"Response from datadog API status_code: {response.status_code}, text: {response.text}")

            verbose_logger.debug(
                "Datadog: Response from datadog API status_code: %s, text: %s",
                response.status_code,
                response.text,
            )

        except Exception as e:
            verbose_logger.exception("Datadog Layer Error - %s\n%s", e, traceback.format_exc())

    async def _log_async_event(self, kwargs, response_obj, start_time, end_time):
        dd_payload: Final = self.create_datadog_logging_payload(
            kwargs=kwargs,
            response_obj=response_obj,
            start_time=start_time,
            end_time=end_time,
        )

View on GitHub (pinned to 77b7c6c40c)