{"record":{"id":"3db7cd7d0da576b1","repo":"BerriAI/litellm","slug":"error-parsing-response-raw-response-text-statu","errorCode":null,"errorMessage":"Error parsing response: {raw_response.text}, status_code={raw_response.status_code}","messagePattern":"Error parsing response: (.+?), status_code=(.+?)","errorType":"http","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/hosted_vllm/rerank/transformation.py","lineNumber":167,"sourceCode":"\n def transform_rerank_response(\n self,\n model: str,\n raw_response: httpx.Response,\n model_response: RerankResponse,\n logging_obj: LiteLLMLoggingObj,\n api_key: str | None = None,\n request_data: dict = {},\n optional_params: dict = {},\n litellm_params: dict = {},\n ) -> RerankResponse:\n \"\"\"\n Process response from Hosted VLLM rerank API\n \"\"\"\n try:\n raw_response_json: Final = raw_response.json()\n except Exception:\n raise ValueError(f\"Error parsing response: {raw_response.text}, status_code={raw_response.status_code}\")\n\n return self._transform_response(raw_response_json)\n\n def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:\n return HostedVLLMRerankError(message=error_message, status_code=status_code, headers=headers)\n\n def _transform_response(self, response: dict) -> RerankResponse:\n # Extract usage information\n usage_data: Final = response.get(\"usage\", {})\n _billed_units: Final = RerankBilledUnits(total_tokens=usage_data.get(\"total_tokens\", 0))\n _tokens: Final = RerankTokens(input_tokens=usage_data.get(\"total_tokens\", 0))\n rerank_meta: Final = RerankResponseMeta(billed_units=_billed_units, tokens=_tokens)\n\n # Extract results\n _results: Final[list[dict] | None] = response.get(\"results\")\n\n if _results is None:\n raise ValueError(f\"No results found in the response={response}\")","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/hosted_vllm/rerank/transformation.py#L149-L185","documentation":"Raised in HostedVLLM RerankConfig.transform_rerank_response when raw_response.json() throws — i.e. the vLLM rerank endpoint returned a body that is not valid JSON. The ValueError embeds the raw text and HTTP status code so you can see what the server actually returned (often an HTML error page, empty body, or a gateway error).","triggerScenarios":"vLLM returns non-JSON: 502/503 HTML from a proxy or gateway in front of vLLM, connection reset producing an empty body, wrong URL hitting a non-API route (server returns plain text/404 page), or the server crashing mid-request.","commonSituations":"Reverse proxy (nginx/traefik/k8s ingress) in front of vLLM returning its own error page; api_base accidentally pointing at the wrong port or path so the response is not the rerank API; vLLM pod restarted/OOM-killed during the request; mixed http/https or auth redirect returning HTML.","solutions":["Read the embedded status_code and body text in the message: 502/503 → proxy/upstream issue, fix or restart the vLLM service; 404 → wrong api_base path; empty body → connection-level failure.","Verify the endpoint manually: curl -X POST /rerank -H 'Content-Type: application/json' -d '{\"model\":\"...\",\"query\":\"q\",\"documents\":[\"a\"]}' and inspect the raw response.","Check api_base points at the vLLM server (and correct port), not a UI or proxy route.","Add retry with backoff at the caller for transient gateway failures."],"exampleFix":"# before\nresult = litellm.rerank(model='hosted_vllm/reranker', query=q, documents=docs,\n api_base='http://vllm:8000')\n# ValueError: Error parsing response: 502 Bad Gateway, status_code=502\n\n# after\nimport time\nfor attempt in range(3):\n try:\n result = litellm.rerank(model='hosted_vllm/reranker', query=q,\n documents=docs, api_base='http://vllm:8000')\n break\n except ValueError as e:\n if attempt == 2:\n raise\n time.sleep(2 ** attempt)\n","handlingStrategy":"retry","validationCode":"null # cannot fully prevent: server/proxy behavior; but can preflight the URL\n\nimport httpx\n\ndef preflight_rerank_endpoint(api_base: str) -> None:\n r = httpx.get(api_base.rstrip(\"/\") + \"/health\", timeout=5)\n r.raise_for_status()","typeGuard":null,"tryCatchPattern":"try:\n result = litellm.rerank(model=\"hosted_vllm/...\", query=q, documents=docs, api_base=base)\nexcept ValueError as e:\n if \"Error parsing response\" in str(e):\n # body/status embedded in message — inspect and retry transient 5xx/gateway errors\n log.warning(\"vLLM rerank non-JSON response: %s\", e)\n backoff_and_retry()\n raise","preventionTips":["Health-check the vLLM endpoint (e.g. /health) before request batches.","Keep proxies/load balancers in front of vLLM from returning HTML error pages — return 503 JSON or fail the connection instead."],"tags":["hosted-vllm","rerank","response-parsing","gateway","http"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}