BerriAI/litellm · error · Exception

An unknown error occurred with the stream

Error message

An unknown error occurred with the stream

What it means

NLP Cloud branch of the main chunk dispatcher: when handle_nlp_cloud_chunk throws and no chunk was successfully emitted yet (sent_first_chunk False, no received_finish_reason), litellm cannot salvage the stream and raises this generic Exception instead of the original parse error.

Source

Thrown at litellm/litellm_core_utils/streaming_handler.py:1261

        elif (
            self.custom_llm_provider and self.custom_llm_provider == "aleph_alpha"
        ):  # aleph alpha doesn't provide streaming
            response_obj = self.handle_aleph_alpha_chunk(chunk)
            completion_obj["content"] = response_obj["text"]
            if response_obj["is_finished"]:
                self.received_finish_reason = response_obj["finish_reason"]
        elif self.custom_llm_provider == "nlp_cloud":
            try:
                response_obj = self.handle_nlp_cloud_chunk(chunk)
                completion_obj["content"] = response_obj["text"]
                if response_obj["is_finished"]:
                    self.received_finish_reason = response_obj["finish_reason"]
            except Exception as e:
                if self.received_finish_reason:
                    raise e
                else:
                    if self.sent_first_chunk is False:
                        raise Exception("An unknown error occurred with the stream")
                    self.received_finish_reason = "stop"
        elif self.custom_llm_provider == "vertex_ai" and not isinstance(chunk, ModelResponseStream):
            chunk = cast(Any, chunk)
            import proto

            if hasattr(chunk, "candidates") is True:
                try:
                    try:
                        completion_obj["content"] = chunk.text
                    except Exception as e:
                        original_exception: Final = e
                        if "Part has no text." in str(e):
                            ## check for function calling
                            function_call: Final = chunk.candidates[0].content.parts[0].function_call

                            args_dict: Final = {}

                            # Check if it's a RepeatedComposite instance

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Treat this as 'NLP Cloud stream died before producing anything' and debug credentials/model first.
2. Retry the request non-streaming to capture NLP Cloud's real error message.
3. Validate the API token and that the model supports streaming.
4. Wrap NLP Cloud calls with fallback provider routing.

try:
    text = "".join(p for p in litellm.completion(model="nlp_cloud/...", stream=True, ...))
except Exception as e:
    if "unknown error occurred with the stream" in str(e):
        text = litellm.completion(model="nlp_cloud/...", stream=False, ...).choices[0].message.content

When it happens

Trigger: The very first NLP Cloud chunk fails to parse (auth error body, non-JSON frame) — later failures either propagate the real error or gracefully finish with 'stop', but a first-chunk failure yields this message.

Common situations: Invalid NLP Cloud token; model unavailable on the account; endpoint region mismatch; response format changes on NLP Cloud.

Related errors

• data_json.get("error")
• chunk
• Braintrust API error: {e.response.text}
• Failed to connect to Braintrust API: {str(e)}
• Failed to retrieve file {file_id} from provider: {str(e)}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/e5ab0c4a398e0ace. Report an issue: GitHub.