BerriAI/litellm · error · ValueError

Failed to decode chunk: {chunk_data}. Error: {e}

Error message

Failed to decode chunk: {chunk_data}. Error: {e}

What it means

Raised by the model_response-based chunk parser at the end of the Bedrock streaming decoders: it converts an already-built ModelResponse (used for fake/non-AWS-event streams) into a streaming chunk, and wraps any failure in ValueError('Failed to decode chunk: {chunk_data}. Error: {e}'). The message embeds both the chunk payload and the underlying error, so the root cause is visible in the text.

Source

Thrown at litellm/llms/bedrock/chat/invoke_handler.py:822

                        arguments=_model_response_tool_call[0].function.arguments,
                    ),
                    index=0,
                )
            processed_chunk: Final = GChunk(
                text=text,
                tool_use=tool_use,
                is_finished=True,
                finish_reason=map_finish_reason(finish_reason=chunk_data.choices[0].finish_reason or ""),
                usage=ChatCompletionUsageBlock(
                    prompt_tokens=chunk_usage.prompt_tokens,
                    completion_tokens=chunk_usage.completion_tokens,
                    total_tokens=chunk_usage.total_tokens,
                ),
                index=0,
            )
            return processed_chunk
        except Exception as e:
            raise ValueError(f"Failed to decode chunk: {chunk_data}. Error: {e}")

    def __next__(self):
        if self.is_done:
            raise StopIteration
        self.is_done = True
        return self._chunk_parser(self.model_response)

    # Async iterator
    def __aiter__(self):
        return self

    async def __anext__(self):
        if self.is_done:
            raise StopAsyncIteration
        self.is_done = True
        return self._chunk_parser(self.model_response)

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Inspect the embedded chunk_data and Error in the ValueError message to see which field was missing or the wrong type.
2. Upgrade litellm - the fake-stream chunk converter has had several fixes for missing usage/finish_reason.
3. Disable fake-stream for that model if you set it explicitly, letting the native event-stream path run.
4. If the provider response is genuinely malformed, log it via LITELLM_LOG=DEBUG and open an issue with the payload.
5. Route via bedrock/converse/<model> to use the Converse schema instead of the invoke fake-stream path.

Example fix

# before
resp = litellm.completion(model="bedrock/<provider>.<model>", messages=msgs)  # fake-stream path errors mid-parse

# after
from litellm.exceptions import BedrockError
try:
    resp = litellm.completion(model="bedrock/converse/<provider>.<model>", messages=msgs)
except ValueError as e:
    msg = str(e)
    if "Failed to decode chunk" in msg:
        logger.error("malformed provider payload: %s", msg)
        raise
    raise

try:
    chunks = list(litellm.completion(model="bedrock/<model>", messages=msgs, stream=True))
except ValueError as e:
    if "Failed to decode chunk" in str(e):
        log.error("malformed chunk payload: %s", e)
        raise
    raise

When it happens

Trigger: Fake-stream mode (fake_stream=True) or provider paths where the completed ModelResponse is re-parsed chunk-wise: failures occur when chunk_data.choices is empty or malformed, finish_reason mapping returns None, or usage fields (prompt/completion/total tokens) are missing or non-numeric.

Common situations: Providers that return empty choices arrays on error, model responses without usage blocks when litellm expects them, or litellm version bugs where the fake-stream re-packing loses fields.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Received streaming error - {e}
• Model needs to be set for bedrock
• BedrockException: Context Window Error - {error_str}
• BedrockException Invalid Authentication - {error_str}
• BedrockException PermissionDeniedError - {error_str}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/434b35e50d2f12a4. Report an issue: GitHub.