BerriAI/litellm · error · VertexAIError

Unexpected error format in mid-stream chunk: {error_data}

Error message

Unexpected error format in mid-stream chunk: {error_data}

What it means

Gemini streaming error detector guard: a chunk contains an 'error' key whose value is not a dict (unexpected provider format), so no status/message can be extracted and a generic 500 VertexAIError naming the payload is raised.

Source

Thrown at litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py:3105

        self.streaming_response = streaming_response
        self.response = response
        self.chunk_type: Literal["valid_json", "accumulated_json"] = "valid_json"
        self.accumulated_json = ""
        self.sent_first_chunk = False
        self.logging_obj = logging_obj
        self.response_headers = response_headers or {}
        self.is_function_call = check_is_function_call(logging_obj)
        self.cumulative_tool_call_index: int = 0
        self.has_seen_tool_calls: bool = False

    @staticmethod
    def _check_streaming_error(chunk: dict) -> None:
        """Detect embedded errors (e.g. 429 RESOURCE_EXHAUSTED) in streaming chunks and raise VertexAIError."""
        if "error" not in chunk:
            return
        error_data: Final = chunk["error"]
        if not isinstance(error_data, dict):
            raise VertexAIError(
                status_code=500,
                message=f"Unexpected error format in mid-stream chunk: {error_data}",
            )
        raw_code = error_data.get("code", 500)
        if raw_code is None:
            raw_code = 500
        try:
            error_code = int(raw_code)
        except (TypeError, ValueError):
            error_code = 500
        error_message: Final = error_data.get("message", "Unknown error")
        error_status: Final = error_data.get("status", "UNKNOWN")
        raise VertexAIError(
            status_code=error_code,
            message=f"{error_status} - {error_message}",
        )

    def _apply_stream_candidates(

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Inspect error_data in the mid-stream chunk; the stream returned an error payload.
2. Check the request for content that triggered a safety or quota error mid-stream.

Example fix

# log error_data to see the stream error detail.

When it happens

Trigger: Triggered when a mid-stream chunk from Vertex AI Gemini contains an unexpected error format.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/226a8f5851a83dbf. Report an issue: GitHub.