BerriAI/litellm · error · ValueError

Failed to decode JSON from chunk: {chunk}

Error message

Failed to decode JSON from chunk: {chunk}

What it means

In the Cohere v1 streaming iterator, each SSE line is json.loads-ed inside chunk_parser; a json.JSONDecodeError (non-JSON line such as an event comment, an HTML fragment, or a garbled chunk) raises ValueError('Failed to decode JSON from chunk: {chunk}') with the offending chunk included. This is the inner parse error — the iterator wrappers (errors 1433/1434) usually rewrap it as RuntimeError.

Source

Thrown at litellm/llms/cohere/common_utils.py:158

                finish_reason = chunk["finish_reason"]

            if "citations" in chunk:
                provider_specific_fields = {"citations": chunk["citations"]}

            returned_chunk: Final = GenericStreamingChunk(
                text=text,
                tool_use=tool_use,
                is_finished=is_finished,
                finish_reason=finish_reason,
                usage=usage,
                index=index,
                provider_specific_fields=provider_specific_fields,
            )

            return returned_chunk

        except json.JSONDecodeError:
            raise ValueError(f"Failed to decode JSON from chunk: {chunk}")

    # Sync iterator
    def __iter__(self):
        return self

    def __next__(self):
        try:
            chunk: Final = self.response_iterator.__next__()
        except StopIteration:
            raise StopIteration
        except ValueError as e:
            raise RuntimeError(f"Error receiving chunk from stream: {e}")

        try:
            return self.convert_str_chunk_to_generic_chunk(chunk=chunk)
        except StopIteration:
            raise StopIteration
        except ValueError as e:

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Look at the chunk embedded in the message — a Cohere error event identifies the real cause (e.g. context length, overloaded).
2. If the chunk is HTML or noise, remove intercepting proxies or fix api_base.
3. Retry the request: mid-stream failures are frequently transient under load.
4. Move to v2 models/command-r family if you are streaming a v2-only model via the v1 path.

try:
    for chunk in cohere_v1_stream:
        consume(chunk)
except ValueError as e:
    if "Failed to decode JSON from chunk" in str(e):
        log_offending_chunk(str(e))
        retry_stream()  # often transient; drop partials unless you track them

When it happens

Trigger: Streaming cohere v1 chat (command models) when a stream line is not JSON: 'data:' lines with non-JSON payloads, an in-stream error event, proxy-injected content, or keep-alive/garbage lines from a middlebox.

Common situations: Cohere sending an error event mid-stream after a partial response; stream corruption through proxies; using a v2 model through the v1 streaming path where event shapes differ.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Error parsing chunk: {e}, Received chunk: {chunk}
• Chunk cannot be parsed as CohereStreamChunk: {e}
• Failed to decode JSON from chunk: {chunk}
• Error parsing chunk: {e}, Received chunk: {chunk}
• Error parsing chunk: {e}, Received chunk: {chunk}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/c81efc1d7b1d2789. Report an issue: GitHub.