{"record":{"id":"bc25e3963afc51ae","repo":"BerriAI/litellm","slug":"error-message-bc25e3","errorCode":null,"errorMessage":"error_message","messagePattern":"error_message","errorType":"exception","errorClass":"OpenAIError","httpStatus":null,"severity":"error","filePath":"litellm/llms/openai/chat/gpt_transformation.py","lineNumber":794,"sourceCode":"        \"\"\"OpenAI-compatible backends (vLLM, sglang) can return an HTTP 200\n        stream whose body carries an error payload, e.g.\n        ``data: {\"error\": {\"message\": \"...\", \"code\": 400}}``.\"\"\"\n        error: Final = chunk.get(\"error\")\n        if not error:\n            return None\n        if not isinstance(error, dict):\n            return str(error), 500\n        message: Final = error.get(\"message\")\n        code: Final = error.get(\"code\")\n        status_code: Final = code if isinstance(code, int) and 400 <= code < 600 else 500\n        return (message if isinstance(message, str) else json.dumps(error)), status_code\n\n    def chunk_parser(self, chunk: dict) -> ModelResponseStream:\n        try:\n            error_details: Final = self._extract_error_from_chunk(chunk)\n            if error_details is not None:\n                error_message, error_status_code = error_details\n                raise OpenAIError(\n                    status_code=error_status_code,\n                    message=error_message,\n                )\n            choices = chunk.get(\"choices\", [])\n            choices = self._map_reasoning_to_reasoning_content(choices)\n\n            kwargs: Final[dict[str, Any]] = {\n                \"id\": chunk.get(\"id\"),\n                \"object\": \"chat.completion.chunk\",\n                \"created\": chunk.get(\"created\"),\n                \"model\": chunk.get(\"model\"),\n                \"choices\": choices,\n            }\n            if \"usage\" in chunk and chunk[\"usage\"] is not None:\n                kwargs[\"usage\"] = chunk[\"usage\"]\n            return ModelResponseStream(**kwargs)\n        except Exception as e:\n            raise e","sourceCodeStart":776,"sourceCodeEnd":812,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/openai/chat/gpt_transformation.py#L776-L812","documentation":"Some OpenAI-compatible backends (vLLM, sglang) return HTTP 200 SSE streams whose individual data chunks carry an error payload like {\"error\": {\"message\": ..., \"code\": 400}}. LiteLLM's chunk_parser extracts that embedded error (message text; code if a 400-599 int, else 500) and raises OpenAIError mid-stream with that status and message.","triggerScenarios":"Streaming from vLLM/sglang (or any openai/ prefixed endpoint) when the server hits an error after the stream starts: model unloaded, request cancelled server-side, max token limit exceeded, or invalid request detected late. The chunk contains an 'error' object rather than 'choices'.","commonSituations":"vLLM backend crashing or preempting requests under load, served model swapped behind a router mid-stream, context length overruns detected after streaming began, or aggregated servers returning in-band errors.","solutions":["Read the message — it is the backend's own error text and states the real cause.","For context-length errors, reduce input tokens/max_tokens; for preemption/OOM, reduce concurrency or increase GPU memory quota.","Retry with backoff for transient backend preemptions; pin the model (e.g. via sticky routing) if a router swaps models mid-stream.","Handle mid-stream failures in your consumer: errors can arrive after several successful chunks."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"def stream_or_fallback(model, messages, **kw):\n    try:\n        return list(litellm.completion(model=model, messages=messages, stream=True, **kw))\n    except litellm.exceptions.OpenAIError as e:\n        if getattr(e, \"status_code\", None) == 400 and kw.get(\"stream\"):\n            # in-band stream error (vLLM/sglang style): retry non-streaming once\n            return [litellm.completion(model=model, messages=messages, **kw)]\n        raise","typeGuard":null,"tryCatchPattern":"chunks = []\ntry:\n    for chunk in litellm.completion(model=\"openai/m\", messages=msgs, stream=True):\n        chunks.append(chunk)\nexcept litellm.exceptions.OpenAIError as e:\n    # error arrived mid-stream with HTTP 200 already sent\n    log.error(\"in-band stream error status=%s: %s\", getattr(e, \"status_code\", \"?\"), e)\n    raise","preventionTips":["Expect errors mid-stream from vLLM/sglang — design consumers to handle partial output plus an exception","Pre-validate token counts to avoid late context-length errors","Use sticky routing when a model router can swap backends mid-stream"],"tags":["openai","streaming","vllm","upstream-error","in-band-error"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}