BerriAI/litellm · warning · Exception

Response from datadog API status_code: {response.status_code

Error message

Response from datadog API status_code: {response.status_code}, text: {response.text}

What it means

After POSTing a batch of log payloads to the Datadog logs intake, the logger expects HTTP 202 (Accepted). Any other status — after raise_for_status() already let non-2xx through as an exception path — produces this Exception carrying the status code and response text. The surrounding except swallows and logs it, so the practical effect is dropped telemetry plus an error log, not a crash of the LLM call.

Source

Thrown at litellm/integrations/datadog/datadog.py:509

                    start_time=start_time,
                    end_time=end_time,
                )

            # Build headers
            headers: Final = {}
            # Add API key if available (required for direct API, optional for agent)
            if self.DD_API_KEY:
                headers["DD-API-KEY"] = self.DD_API_KEY

            response: Final = self.sync_client.post(
                url=self.intake_url,
                json=dd_payload,
                headers=headers,
            )

            response.raise_for_status()
            if response.status_code != 202:
                raise Exception(f"Response from datadog API status_code: {response.status_code}, text: {response.text}")

            verbose_logger.debug(
                "Datadog: Response from datadog API status_code: %s, text: %s",
                response.status_code,
                response.text,
            )

        except Exception as e:
            verbose_logger.exception("Datadog Layer Error - %s\n%s", e, traceback.format_exc())

    async def _log_async_event(self, kwargs, response_obj, start_time, end_time):
        dd_payload: Final = self.create_datadog_logging_payload(
            kwargs=kwargs,
            response_obj=response_obj,
            start_time=start_time,
            end_time=end_time,
        )

View on GitHub (pinned to 6c2dcb801b)

Solutions

Check verbose_logger output for the exact status/text pair: 403 -> fix DD_API_KEY, 404/400 -> fix DD_SITE, 429 -> batch/rate limits
Verify the key/site pair by curling the intake directly with a minimal payload
Rotate or re-issue the API key in Datadog and update the proxy environment, then restart

Example fix

# before
DD_SITE=datadoghq.com   # generic site, org is on us5

# after
DD_SITE=us5.datadoghq.com

Defensive patterns

Strategy: try-catch

Validate before calling

import httpx, os

def intake_ok() -> bool:
    r = httpx.post(
        f"https://http-intake.logs.{os.getenv('DD_SITE')}/api/v2/logs",
        headers={"DD-API-KEY": os.environ["DD_API_KEY"]},
        json=[{"message": "smoke"}],
    )
    return r.status_code == 202

Try / catch

# litellm already swallows and logs this; monitor for it instead:
# grep 'Datadog Layer Error' in proxy logs and alert on repeats;
# 403 -> key, 404/400 -> site, 429 -> throughput

Prevention

Alert on repeated 'Datadog Layer Error' log lines — logs are silently dropped otherwise
Smoke-test the key/site pair with a curl POST before deploying
Rotate Datadog keys on a schedule and update the proxy env in the same change

When it happens

Trigger: Datadog intake returns non-202 for the batch: 403 Forbidden (wrong/revoked API key), 400 (malformed payload or wrong site), 404 (site mismatch, e.g. DD_SITE=us5 but key belongs to another region), or 429 rate limiting.

Common situations: DD_SITE set to the wrong region for the org; API key rotated in Datadog but not in the proxy env; large bursts exceeding intake limits; clock/payload schema drift after upgrading litellm's Datadog serializer.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/ab7429b91d75cbcf. Report an issue: GitHub.