{"record":{"id":"5ea8767ff9793fff","repo":"BerriAI/litellm","slug":"raw-response-text-5ea876","errorCode":null,"errorMessage":"{raw_response.text}","messagePattern":"\\{raw_response\\.text\\}","errorType":"http","errorClass":"HuggingFaceError","httpStatus":null,"severity":"error","filePath":"litellm/llms/huggingface/rerank/transformation.py","lineNumber":181,"sourceCode":"        request_body.update(optional_rerank_params)\n\n        return request_body\n\n    def transform_rerank_response(\n        self,\n        model: str,\n        raw_response: httpx.Response,\n        model_response: RerankResponse,\n        logging_obj: LoggingClass,\n        api_key: str | None = None,\n        request_data: dict = {},\n        optional_params: dict = {},\n        litellm_params: dict = {},\n    ) -> RerankResponse:\n        try:\n            raw_response_json: Final[HuggingFaceRerankResponseList] = raw_response.json()\n        except Exception:\n            raise HuggingFaceError(\n                message=getattr(raw_response, \"text\", str(raw_response)),\n                status_code=getattr(raw_response, \"status_code\", 500),\n            )\n\n        # Use standard litellm token counter for proper token estimation\n        input_text = request_data.get(\"query\", \"\")\n        try:\n            # Calculate tokens for the raw response JSON string\n            response_text: Final = str(raw_response_json)\n            estimated_output_tokens = token_counter(model=model, text=response_text)\n\n            # Calculate input tokens from query and documents\n            query: Final = request_data.get(\"query\", \"\")\n            documents: Final = request_data.get(\"texts\", [])\n\n            # Convert documents to string if they're not already\n            documents_text = \"\"\n            for doc in documents:","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/huggingface/rerank/transformation.py#L163-L199","documentation":"Raised when the HuggingFace rerank endpoint's response body cannot be parsed as JSON. The message is the raw body text (or str(response) as fallback) and the status defaults to 500 when unavailable — typically an HTML error page or empty body from a gateway/proxy in front of the rerank service.","triggerScenarios":"Self-hosted reranker (e.g. Infinity/text-embeddings-inference) behind a reverse proxy returning HTML 502, wrong api_base path returning the web UI, or a crashed worker returning an empty body.","commonSituations":"Kubernetes ingress/nginx returning error pages during rollout, or a port/path mismatch so the request never reaches the rerank handler.","solutions":["Inspect the raw body in the message to identify the proxy/gateway source.","Fix api_base to the actual rerank route; confirm with curl.","Add retry-with-backoff for transient 5xx gateway responses."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"def endpoint_serves_json(url: str) -> bool:\n    import httpx\n    try:\n        return 'json' in httpx.get(url, timeout=5).headers.get('content-type', '')\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    result = litellm.rerank(model=model, query=q, documents=docs, api_base=base)\nexcept litellm.llms.huggingface.common_utils.HuggingFaceError as e:\n    if '<html' in str(e).lower():\n        raise RuntimeError(f'{base} is fronted by an HTML error page; check deployment') from e\n    raise","preventionTips":["Health-check self-hosted rerankers (JSON /health or /models) before traffic","Pin deployment ports/paths in config reviewed at deploy time"],"tags":["huggingface","rerank","response-format","gateway","json"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}