BerriAI/litellm · error · AnthropicError

Unable to get json response - {e}, Original Response: {raw_r

Error message

Unable to get json response - {e}, Original Response: {raw_response.text}

What it means

The HTTP call returned, but raw_response.json() failed — the body is not JSON. LiteLLM includes the original response text in the error so you can see exactly what came back. Typical culprits: an HTML error page from a gateway/proxy (502/504), an empty body from a crashed upstream, or a plain-text error from a misrouted endpoint.

Source

Thrown at litellm/llms/anthropic/chat/transformation.py:2490

        litellm_params: dict,
        encoding: Any,
        api_key: str | None = None,
        json_mode: bool | None = None,
    ) -> ModelResponse:
        ## LOGGING
        logging_obj.post_call(
            input=messages,
            api_key=api_key,
            original_response=raw_response.text,
            additional_args={"complete_input_dict": request_data},
        )

        ## RESPONSE OBJECT
        try:
            completion_response: Final = raw_response.json()
        except Exception as e:
            response_headers: Final = getattr(raw_response, "headers", None)
            raise AnthropicError(
                message=f"Unable to get json response - {e}, Original Response: {raw_response.text}",
                status_code=raw_response.status_code,
                headers=response_headers,
            )

        prefix_prompt: Final = self.get_prefix_prompt(messages=messages)
        speed: Final = optional_params.get("speed")
        tool_name_reverse_map: dict[str, str] | None = None
        if isinstance(litellm_params, dict):
            _candidate: Final = litellm_params.get(ANTHROPIC_TOOL_NAME_REVERSE_MAP_KEY)
            if isinstance(_candidate, dict):
                tool_name_reverse_map = _candidate

        model_response = self.transform_parsed_response(
            completion_response=completion_response,
            raw_response=raw_response,
            model_response=model_response,
            json_mode=json_mode,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read 'Original Response:' in the message — it reveals the actual body (usually an HTML 502/504 page naming the failing hop).
2. Fix or restart the gateway/proxy at that hop; verify the backend URL it forwards to.
3. If the body is empty, check upstream timeouts and connection resets between proxy and api.anthropic.com.
4. Add a retry with backoff for transient gateway failures (502/503/504).
5. Confirm ANTHROPIC_API_BASE is correct and not pointing at an OpenAI-only endpoint.

Example fix

# before
resp = litellm.completion(model="anthropic/claude-sonnet-4-5", messages=msgs)

# after
from litellm.exceptions import AnthropicError
import time
for attempt in range(3):
    try:
        resp = litellm.completion(
            model="anthropic/claude-sonnet-4-5", messages=msgs,
            num_retries=2,
        )
        break
    except AnthropicError as e:
        if "Unable to get json response" in str(e) and attempt < 2:
            time.sleep(2 ** attempt)
            continue
        raise

from litellm.exceptions import AnthropicError
for attempt in range(3):
    try:
        resp = litellm.completion(model=MODEL, messages=messages)
        break
    except AnthropicError as e:
        if "Unable to get json response" in str(e):
            log.error("non-JSON body (attempt %d): %.200s", attempt, e.message)
            time.sleep(2 ** attempt)
            continue
        raise

When it happens

Trigger: Non-streaming anthropic/ call where the response body is HTML (nginx '502 Bad Gateway', Cloudflare challenge page), empty (connection closed after headers), or plain text — most often because api_base points at a proxy/gateway that failed to reach Anthropic.

Common situations: Custom ANTHROPIC_API_BASE to an internal gateway that intermittently 502s; DNS/tls interceptors returning block pages; LiteLLM proxy misrouting to a dead backend; corporate proxies returning text errors.

Related errors

• Failed to decode JSON from chunk: {chunk}
• Failed to parse Clarifai response: {e}
• Error parsing response: {raw_response.text}, status_code={ra
• Failed to transform Braintrust response: {str(e)}
• Error apply_db_fixes: {str(e)}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/5bdc055814937d9d. Report an issue: GitHub.