{"record":{"id":"ae727c0b9155c1ca","repo":"BerriAI/litellm","slug":"raw-response-text-ae727c","errorCode":null,"errorMessage":"{raw_response.text}","messagePattern":"\\{raw_response\\.text\\}","errorType":"http","errorClass":"BaseLLMException","httpStatus":null,"severity":"error","filePath":"litellm/llms/nvidia_nim/rerank/transformation.py","lineNumber":296,"sourceCode":"                }\n            ]\n        }\n\n        LiteLLM expects (RerankResponse):\n        {\n            \"results\": [\n                {\n                    \"index\": 0,\n                    \"relevance_score\": 0.123,\n                    \"document\": {\"text\": \"...\"}  # optional\n                }\n            ]\n        }\n        \"\"\"\n        try:\n            raw_response_json: Final = raw_response.json()\n        except Exception:\n            raise BaseLLMException(\n                status_code=raw_response.status_code,\n                message=raw_response.text,\n                headers=raw_response.headers,\n            )\n\n        # Parse as NvidiaNimRerankResponse\n        nvidia_response: Final[NvidiaNimRerankResponse] = raw_response_json\n\n        # Transform Nvidia NIM response to LiteLLM format\n        results: Final[list[RerankResponseResult]] = []\n        rankings: Final = nvidia_response.get(\"rankings\", [])\n\n        # Get original documents from request if we need to include them\n        original_passages: Final[list[NvidiaNimPassageObject]] = request_data.get(\"passages\", [])\n\n        for ranking in rankings:\n            result_item: RerankResponseResult = {\n                \"index\": ranking[\"index\"],","sourceCodeStart":278,"sourceCodeEnd":314,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/nvidia_nim/rerank/transformation.py#L278-L314","documentation":"Raised by litellm's Nvidia NIM rerank response transformer when raw_response.json() fails: the NIM /v1/ranking endpoint returned a non-JSON body. The exception message is the raw response text, and the upstream status code and headers are propagated on the BaseLLMException.","triggerScenarios":"Reranking against nvidia_nim/* when the endpoint returns HTML/plain text: invalid NVIDIA_NIM_API_KEY (401 HTML), wrong api_base pointing at a UI page, a self-hosted NIM container that is down (502 from ingress), or model not deployed on the local NIM.","commonSituations":"Wrong api_base for self-hosted NIM (missing /v1 suffix or pointing at the docs UI), expired nvapi key against build.nvidia.com, or ingress/gateway errors in Kubernetes.","solutions":["Read the message/status_code — the literal upstream body identifies auth vs routing vs outage.","Confirm api_base is the NIM ranking route (e.g. http://host:8000/v1/ranking or the hosted equivalent).","For self-hosted NIM, verify the container is up and the reranking model is loaded (curl the /v1/models endpoint).","Refresh NVIDIA_NIM_API_KEY if targeting the hosted endpoint."],"exampleFix":"# before\nlitellm.rerank(model=\"nvidia_nim/nv-rerankqa-mistral-4b-v3\", query=q, documents=docs)  # wrong base\n\n# after\nlitellm.rerank(\n    model=\"nvidia_nim/nv-rerankqa-mistral-4b-v3\",\n    query=q,\n    documents=docs,\n    api_base=\"http://nim-host:8000/v1/ranking\",\n    api_key=os.environ[\"NVIDIA_NIM_API_KEY\"],\n)","handlingStrategy":"try-catch","validationCode":"import httpx, os\nbase = \"http://nim-host:8000\"\nresp = httpx.get(f\"{base}/v1/models\", timeout=5)  # preflight: is the NIM up?\nassert resp.status_code == 200, f\"NIM unreachable at {base}\"","typeGuard":null,"tryCatchPattern":"from litellm.exceptions import APIError\ntry:\n    res = litellm.rerank(model=\"nvidia_nim/nv-rerankqa-mistral-4b-v3\", query=q, documents=docs)\nexcept APIError as e:\n    status = getattr(e, \"status_code\", None)\n    if status == 401:\n        raise RuntimeError(\"Invalid NVIDIA_NIM_API_KEY\") from e\n    if status and status >= 500:\n        return retry_with_backoff()  # gateway/nim down\n    raise","preventionTips":["Health-check self-hosted NIM /v1/models before routing rerank traffic.","Keep api_base in config, not code, so environments differ cleanly.","Log the raw body in this error — it identifies routing vs auth failures."],"tags":["nvidia-nim","rerank","json-parse","api-base"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}