BerriAI/litellm · error · ValueError

Unable to parse response. Original response: {chunk}

Error message

Unable to parse response. Original response: {chunk}

What it means

The AI21 handler is fake-streaming: it decodes one JSON body and indexes data_json['completions'][0]['data']['text']. Any deviation — missing 'completions', empty list, nested key rename — throws and is converted to this ValueError including the raw body.

Source

Thrown at litellm/litellm_core_utils/streaming_handler.py:513

                "finish_reason": finish_reason,
            }
        except Exception as e:
            raise e

    def handle_ai21_chunk(self, chunk):  # fake streaming
        chunk = chunk.decode("utf-8")
        data_json: Final[_Ai21StreamData] = json.loads(chunk)
        try:
            text: Final = data_json["completions"][0]["data"]["text"]
            is_finished: Final = True
            finish_reason: Final = "stop"
            return {
                "text": text,
                "is_finished": is_finished,
                "finish_reason": finish_reason,
            }
        except Exception:
            raise ValueError(f"Unable to parse response. Original response: {chunk}")

    def handle_maritalk_chunk(self, chunk):  # fake streaming
        chunk = chunk.decode("utf-8")
        data_json: Final[_MaritalkStreamData] = json.loads(chunk)
        try:
            text: Final = data_json["answer"]
            is_finished: Final = True
            finish_reason: Final = "stop"
            return {
                "text": text,
                "is_finished": is_finished,
                "finish_reason": finish_reason,
            }
        except Exception:
            raise ValueError(f"Unable to parse response. Original response: {chunk}")

    def handle_nlp_cloud_chunk(self, chunk):
        text = ""

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read the raw chunk embedded in the message — it shows the actual AI21 response.
2. Verify the AI21 API key and model name (e.g. j2-ultra) still work with a non-streaming call first.
3. Migrate to AI21's current models (jamba) which use a different handler.
4. Pin/upgrade litellm to a version matching the AI21 API you target.

try:
    text = "".join(p for p in litellm.completion(model="ai21/j2-ultra", stream=True, ...))
except ValueError as e:
    if "Unable to parse response" in str(e):
        fallback = litellm.completion(model="ai21/j2-ultra", stream=False, ...)

When it happens

Trigger: Calling AI21 (j2-era) models with stream=True when the API returns an error body or a schema without completions[0].data.text (e.g. auth error JSON, model deprecation response).

Common situations: Using legacy AI21 Jurassic-2 endpoints after schema changes or deprecation; invalid API key producing a JSON error body that still parses as JSON; region mismatch.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Braintrust API error: {e.response.text}
• Failed to connect to Braintrust API: {str(e)}
• api_base is required for Pydantic AI agents
• Unexpected responses stream payload
• Stream ended without a completed response

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/d487fad2666b3547. Report an issue: GitHub.