{"record":{"id":"ca656d5af820adb8","repo":"BerriAI/litellm","slug":"raw-response-text-ca656d","errorCode":null,"errorMessage":"raw_response.text","messagePattern":"raw_response\\.text","errorType":"exception","errorClass":"TritonError","httpStatus":null,"severity":"error","filePath":"litellm/llms/triton/embedding/transformation.py","lineNumber":84,"sourceCode":"                }\n            ]\n        }\n\n    def transform_embedding_response(\n        self,\n        model: str,\n        raw_response: httpx.Response,\n        model_response: EmbeddingResponse,\n        logging_obj: LiteLLMLoggingObj,\n        api_key: str | None = None,\n        request_data: dict = {},\n        optional_params: dict = {},\n        litellm_params: dict = {},\n    ) -> EmbeddingResponse:\n        try:\n            raw_response_json: Final = raw_response.json()\n        except Exception:\n            raise TritonError(message=raw_response.text, status_code=raw_response.status_code)\n\n        _embedding_output: Final = []\n\n        _outputs: Final = raw_response_json[\"outputs\"]\n        for output in _outputs:\n            _shape = output[\"shape\"]\n            _data = output[\"data\"]\n            _split_output_data = self.split_embedding_by_shape(_data, _shape)\n\n            for idx, embedding in enumerate(_split_output_data):\n                _embedding_output.append(\n                    {\n                        \"object\": \"embedding\",\n                        \"index\": idx,\n                        \"embedding\": embedding,\n                    }\n                )\n","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/triton/embedding/transformation.py#L66-L102","documentation":"The Triton embedding handler parses the HTTP response with raw_response.json(); if the body is not JSON it raises TritonError containing raw_response.text and status_code. For embeddings, Triton's /infer endpoint should return {'outputs': [{'shape': [...], 'data': [...]}]}; a non-JSON body means the server errored out (unknown model, wrong input name, load failure) and returned an error page/plain text instead.","triggerScenarios":"Calling litellm.embedding(model=\"triton/...\", input=[...]) with api_base at a Triton /infer endpoint where the model is missing/misconfigured (404 'unavailable model'), input tensor names mismatch, or an ingress returns HTML 502; any non-JSON reply triggers this.","commonSituations":"Embedding models (e.g. tensorrt/onnx encoders) whose config.pbtxt inputs don't match the payload; K8s ingress errors during rollout; using the server root instead of the full /v2/models/<m>/infer path so Triton returns a docs page.","solutions":["Read the TritonError message — it embeds Triton's raw error text (usually names the unknown model or input).","Confirm the model is loaded and ready: curl http://<host>:8000/v2/models/<model>/ready.","Check api_base is the full /v2/models/<model>/infer path (satisfying the /infer suffix contract).","Align config.pbtxt input names/shapes (e.g. 'input_ids'/'INPUT_TEXT') with what the embedding request sends."],"exampleFix":"# before\nresp = litellm.embedding(\n    model=\"triton/my-embed\",\n    input=[\"hello world\"],\n    api_base=\"http://triton:8000/v2/models/my-embed\",  # missing /infer\n)\n# TritonError: [404] ... non-JSON body ...\n\n# after\nresp = litellm.embedding(\n    model=\"triton/my-embed\",\n    input=[\"hello world\"],\n    api_base=\"http://triton:8000/v2/models/my-embed/infer\",\n)","handlingStrategy":"try-catch","validationCode":"import httpx\n\ndef triton_embedding_endpoint_ok(base: str, model: str) -> bool:\n    \"\"\"Ready probe + URL shape check before embedding calls.\"\"\"\n    if not (base.endswith(\"/infer\") or base.endswith(\"/generate\")):\n        return False\n    try:\n        return httpx.get(f\"{base.rsplit('/v2/', 1)[0]}/v2/models/{model}/ready\", timeout=2).status_code == 200\n    except httpx.HTTPError:\n        return False","typeGuard":null,"tryCatchPattern":"try:\n    resp = litellm.embedding(\n        model=\"triton/my-embed\", input=texts,\n        api_base=\"http://triton:8000/v2/models/my-embed/infer\",\n    )\nexcept Exception as e:\n    if getattr(e, \"status_code\", None) is not None:  # TritonError with raw body\n        logger.error(\"triton embedding failed [%s]: %s\", e.status_code, e)\n        raise RuntimeError(\"check triton model readiness/config\") from e\n    raise","preventionTips":["Always use the full /v2/models/<model>/infer path for embedding endpoints.","Probe model readiness before batch embedding jobs.","Validate config.pbtxt embedding input/output tensor names against LiteLLM's payload once, in a staging test."],"tags":["triton","embeddings","self-hosted","json-decode","response-parsing","litellm"],"backgroundTag":"invalid-json-response","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}