{"record":{"id":"3db7cd7d0da576b1","repo":"BerriAI/litellm","slug":"error-parsing-response-raw-response-text-statu","errorCode":null,"errorMessage":"Error parsing response: {raw_response.text}, status_code={raw_response.status_code}","messagePattern":"Error parsing response: (.+?), status_code=(.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/hosted_vllm/rerank/transformation.py","lineNumber":167,"sourceCode":"\n    def transform_rerank_response(\n        self,\n        model: str,\n        raw_response: httpx.Response,\n        model_response: RerankResponse,\n        logging_obj: LiteLLMLoggingObj,\n        api_key: str | None = None,\n        request_data: dict = {},\n        optional_params: dict = {},\n        litellm_params: dict = {},\n    ) -> RerankResponse:\n        \"\"\"\n        Process response from Hosted VLLM rerank API\n        \"\"\"\n        try:\n            raw_response_json: Final = raw_response.json()\n        except Exception:\n            raise ValueError(f\"Error parsing response: {raw_response.text}, status_code={raw_response.status_code}\")\n\n        return self._transform_response(raw_response_json)\n\n    def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:\n        return HostedVLLMRerankError(message=error_message, status_code=status_code, headers=headers)\n\n    def _transform_response(self, response: dict) -> RerankResponse:\n        # Extract usage information\n        usage_data: Final = response.get(\"usage\", {})\n        _billed_units: Final = RerankBilledUnits(total_tokens=usage_data.get(\"total_tokens\", 0))\n        _tokens: Final = RerankTokens(input_tokens=usage_data.get(\"total_tokens\", 0))\n        rerank_meta: Final = RerankResponseMeta(billed_units=_billed_units, tokens=_tokens)\n\n        # Extract results\n        _results: Final[list[dict] | None] = response.get(\"results\")\n\n        if _results is None:\n            raise ValueError(f\"No results found in the response={response}\")","sourceCodeStart":149,"sourceCodeEnd":185,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/hosted_vllm/rerank/transformation.py#L149-L185","documentation":"After vLLM answers a rerank request, transform_rerank_response() calls raw_response.json(); if the body is not valid JSON it raises ValueError embedding the raw body text and the HTTP status code for diagnosis. Common non-JSON bodies include HTML error pages from proxies, plain-text gateway errors (502/503), or empty bodies from crashed connections. The status_code in the message is the key diagnostic.","triggerScenarios":"api_base pointing at a server that answers /rerank with HTML (a reverse proxy 404/502 page, an auth middleware); a gateway timeout returning a plain-text error; vLLM crashing mid-response; hitting a route that is not the rerank endpoint.","commonSituations":"Base URL misconfigured (missing port, wrong path, pointing at a frontend instead of vLLM); ingress/load-balancer intercepting the request; vLLM not started with a rerank-capable model so an error page is returned; TLS termination returning an HTML challenge.","solutions":["Read the embedded status_code and body snippet — an HTML 404 page means wrong path, a 502/503 means gateway/upstream trouble","Reproduce outside LiteLLM: curl -X POST <api_base>/rerank with the same payload and inspect the raw response","Confirm the vLLM server exposes /rerank and was launched with a reranker model (e.g. jina-reranker, bge-reranker)","Fix the base URL/proxy so JSON from vLLM reaches the client; retry only on transient 5xx"],"exampleFix":"# before\nresult = litellm.rerank(model=\"hosted_vllm/jina-reranker\", api_base=\"http://gateway.internal\", query=q, documents=docs)\n\n# after — point directly at the vLLM instance serving /rerank\nresult = litellm.rerank(model=\"hosted_vllm/jina-reranker\", api_base=\"http://vllm-rerank.internal:8000\", query=q, documents=docs)","handlingStrategy":"try-catch","validationCode":"import httpx\n\ndef rerank_endpoint_speaks_json(base: str) -> bool:\n    try:\n        r = httpx.post(f\"{base.rstrip('/')}/rerank\", json={\"model\": \"x\", \"query\": \"q\", \"documents\": [\"d\"]}, timeout=5)\n        r.json()\n        return True\n    except ValueError:\n        return False","typeGuard":null,"tryCatchPattern":"Catch ValueError around litellm.rerank and parse the embedded status code from the message; treat 5xx/gateway snippets in the body as transient (retry with backoff, bounded attempts) and anything else as configuration failure (halt and alert with the raw body excerpt). Never retry blindly — the message tells you which class of failure occurred.","preventionTips":["Health-check the /rerank route (expect JSON) during service startup and after deploys","Point api_base directly at the vLLM instance; keep proxies transparent or exclude /rerank from HTML error pages","Log the full ValueError message — it embeds the exact non-JSON body and status"],"tags":["hosted-vllm","rerank","json-parsing","gateway-error","litellm"],"backgroundTag":"invalid-json-response","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}