BerriAI/litellm · error · RuntimeError

Error parsing chunk: , Received chunk

Error message

Error parsing chunk: {e},
Received chunk: {chunk}

What it means

RuntimeError raised by the Cohere sync streaming iterator when convert_str_chunk_to_generic_chunk raises ValueError — i.e. the raw line was received fine but could not be turned into a GenericStreamingChunk (JSON decode of the payload, or unexpected event shape). Both the original error and the full received line are included in the message, making the offending SSE event directly visible.

Solutions

1. Read the 'Received chunk:' portion of the message to see the exact malformed event and its source
2. Upgrade litellm to pick up current Cohere v2 streaming event handling
3. Remove/fix intermediary proxies that modify SSE, and disable response buffering
4. Reproduce with stream=False to check whether the account/endpoint returns an error at all

stream = litellm.completion(model="cohere/command-r", messages=messages, stream=True)
try:
    for chunk in stream:
        handle(chunk)
except RuntimeError as e:
    if "Error parsing chunk" in str(e):
        # message includes 'Received chunk:' — log it and degrade to non-streaming
        log.warning("Cohere stream malformed: %s", e)
        resp = litellm.completion(model="cohere/command-r", messages=messages)
        handle(resp)
    else:
        raise

When it happens

Trigger: A Cohere stream line that is not valid JSON or lacks the fields litellm expects — gateway-injected text lines, mid-stream plain-error events, or a Cohere streaming-format change not handled by the installed litellm version.

Common situations: Proxy-altered SSE payloads; litellm/Cohere version skew after event schema updates (e.g. new event types on v2); truncated final events after connection issues; endpoints that close streams with a plain-text error.

Related errors

• Failed to decode JSON from chunk
• Chunk cannot be parsed as CohereStreamChunk
• Chat provider: Empty parsed_chunk
• Chat provider: Invalid chunk type
• Chat provider: Invalid function argument delta

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/9c5cbcf31abf2a02. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/cohere/common_utils.py:177

    # Sync iterator
    def __iter__(self):
        return self

    def __next__(self):
        try:
            chunk: Final = self.response_iterator.__next__()
        except StopIteration:
            raise StopIteration
        except ValueError as e:
            raise RuntimeError(f"Error receiving chunk from stream: {e}")

        try:
            return self.convert_str_chunk_to_generic_chunk(chunk=chunk)
        except StopIteration:
            raise StopIteration
        except ValueError as e:
            raise RuntimeError(f"Error parsing chunk: {e},\nReceived chunk: {chunk}")

    def convert_str_chunk_to_generic_chunk(self, chunk: str) -> GenericStreamingChunk:
        """
        Convert a string chunk to a GenericStreamingChunk

        Note: This is used for Cohere pass through streaming logging
        """
        str_line = chunk
        if isinstance(chunk, bytes):  # Handle binary data
            str_line = chunk.decode("utf-8")  # Convert bytes to string
            index: Final = str_line.find("data:")
            if index != -1:
                str_line = str_line[index:]

        data_json: Final = json.loads(str_line)
        return self.chunk_parser(chunk=data_json)

    # Async iterator

View on GitHub (pinned to 77b7c6c40c)