BerriAI/litellm · error · BedrockError

AgentCore: Failed to read/parse JSON response body

Error message

AgentCore: Failed to read/parse JSON response body: {e}

What it means

During synchronous AgentCore streaming, if the response Content-Type is application/json (the agent returned synchronously instead of via SSE), the handler reads the body with response.read() and json.loads it. Failure of either step raises BedrockError carrying the upstream status code — the endpoint promised JSON but the bytes were empty or unparseable.

Solutions

  1. Catch BedrockError and branch on status_code — 5xx/truncation cases are retryable with backoff
  2. Increase client/stream timeouts and disable aggressive proxy buffering for amazonaws.com
  3. Retry the request once or twice with exponential backoff; if it persists, reproduce with a minimal prompt
  4. Update LiteLLM — robustness of the JSON-instead-of-SSE path has improved across releases
Defensive patterns

Strategy: retry

Try / catch

from litellm.exceptions import BedrockError

for attempt in range(3):
    try:
        chunks = list(litellm.completion(model=agentcore_model, messages=msgs, stream=True))
        break
    except BedrockError as e:
        if e.status_code is not None and e.status_code >= 500 and attempt < 2:
            time.sleep(2 ** attempt)
            continue
        raise

Prevention

When it happens

Trigger: AgentCore returns Content-Type: application/json with an empty body (common on some error statuses); the connection drops mid-body so read()/loads see truncated bytes; an intermediate proxy emits a JSON content-type on a non-JSON error page.

Common situations: Long-running agents hitting idle or proxy timeouts that truncate the body; intermittent 5xx from the AgentCore runtime; API gateways rewriting error responses; older LiteLLM versions with weaker JSON-mode handling.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/88627320296f74bf. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/bedrock/chat/agentcore/transformation.py:687

        # LOGGING
        logging_obj.post_call(
            input=messages,
            api_key="",
            original_response="first stream response received",
            additional_args={"complete_input_dict": data},
        )

        # Check if response is JSON (agent used sync return) instead of SSE
        content_type: Final = response.headers.get("content-type", "").lower()
        if "application/json" in content_type:
            verbose_logger.debug(
                "AgentCore streaming: received JSON response instead of SSE, converting to single-chunk stream"
            )
            try:
                body: Final = response.read()
                response_json: Final = json.loads(body)
            except (json.JSONDecodeError, Exception) as e:
                raise BedrockError(
                    status_code=response.status_code,
                    message=f"AgentCore: Failed to read/parse JSON response body: {e}",
                )
            parsed: Final = self._parse_json_response(response_json)

            def _json_as_sync_stream():
                # Content chunk
                content_chunk: Final = ModelResponseStream(
                    id=f"chatcmpl-{uuid.uuid4()}",
                    created=0,
                    model=model,
                    object="chat.completion.chunk",
                )
                content_chunk.choices = [
                    StreamingChoices(
                        finish_reason=None,
                        index=0,
                        delta=Delta(content=parsed["content"], role="assistant"),

View on GitHub (pinned to 77b7c6c40c)