BerriAI/litellm · error · Exception

data_json.get("error")

Error message

data_json.get("error")

What it means

In the Predibase stream chunk handler, when a parsed SSE payload contains a top-level 'error' key the handler raises Exception with that value, surfacing the upstream error from the Predibase inference server.

Source

Thrown at litellm/litellm_core_utils/streaming_handler.py:484

                chunk = chunk.decode("utf-8")  # DO NOT REMOVE this: This is required for HF inference API + Streaming
            text = ""
            is_finished = False
            finish_reason = ""
            print_verbose(f"chunk: {chunk}")
            if chunk.startswith("data:"):
                data_json: Final[_PredibaseStreamData] = json.loads(chunk[5:])
                print_verbose(f"data json: {data_json}")
                if "token" in data_json and "text" in data_json["token"]:
                    text = data_json["token"]["text"]
                if data_json.get("details", False) and data_json["details"].get("finish_reason", False):
                    is_finished = True
                    finish_reason = data_json["details"]["finish_reason"]
                elif data_json.get("generated_text", False):  # if full generated text exists, then stream is complete
                    text = ""  # don't return the final bos token
                    is_finished = True
                    finish_reason = "stop"
                elif data_json.get("error", False):
                    raise Exception(data_json.get("error"))
                return {
                    "text": text,
                    "is_finished": is_finished,
                    "finish_reason": finish_reason,
                }
            elif "error" in chunk:
                raise ValueError(chunk)
            return {
                "text": text,
                "is_finished": is_finished,
                "finish_reason": finish_reason,
            }
        except Exception as e:
            raise e

    def handle_ai21_chunk(self, chunk):  # fake streaming
        chunk = chunk.decode("utf-8")
        data_json: Final[_Ai21StreamData] = json.loads(chunk)

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read the exception message — it echoes Predibase's own error text (auth vs model vs capacity).
2. Verify the Predibase API token and the adapter/deployment id in your call.
3. Confirm the deployment is running and the model string matches Predibase's docs for litellm.
4. Retry after the deployment is healthy; this is a server-side error frame, not a parsing bug.

try:
    for part in litellm.completion(model="predibase/...", stream=True, ...):
        process(part)
except Exception as e:
    log.error("Predibase stream error frame: %s", e)
    raise UpstreamProviderError("predibase") from e

When it happens

Trigger: Streaming a completion with custom_llm_provider='predibase' where the server emits {"error": ...} frames — invalid model/adapter id, expired Predibase token, or a request the deployed LLM rejects mid-stream.

Common situations: Wrong adapter_id or model name for a Predibase deployment; revoked API key; Predibase deployment restarting/terminated; payload exceeds server limits.

Related errors

• chunk
• An unknown error occurred with the stream
• Ollama Error - {chunk}
• error_message
• Braintrust API error: {e.response.text}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/a4b6387d469e72b6. Report an issue: GitHub.