{"record":{"id":"7090bb30ac22824a","repo":"BerriAI/litellm","slug":"failed-to-parse-raw-azure-embedding-response-jso","errorCode":null,"errorMessage":"Failed to parse raw Azure embedding response: {json_error}","messagePattern":"Failed to parse raw Azure embedding response: (.+?)","errorType":"exception","errorClass":"AzureOpenAIError","httpStatus":null,"severity":"error","filePath":"litellm/llms/azure/azure.py","lineNumber":698,"sourceCode":"            raw_response: Final = await openai_aclient.embeddings.with_raw_response.create(**data, timeout=timeout)\n            headers: Final = dict(raw_response.headers)\n\n            # Convert json.JSONDecodeError to AzureOpenAIError for two critical reasons:\n            #\n            # 1. ROUTER BEHAVIOR: The router relies on exception.status_code to determine cooldown logic:\n            #    - JSONDecodeError has no status_code → router skips cooldown evaluation\n            #    - AzureOpenAIError has status_code → router properly evaluates for cooldown\n            #\n            # 2. CONNECTION CLEANUP: When response.parse() throws JSONDecodeError, the response\n            #    body may not be fully consumed, preventing httpx from properly returning the\n            #    connection to the pool. By catching the exception and accessing raw_response.status_code,\n            #    we trigger httpx's internal cleanup logic. Without this:\n            #    - parse() fails → JSONDecodeError bubbles up → httpx never knows response was acknowledged → connection leak\n            #    This completely eliminates \"Unclosed connection\" warnings during high load.\n            try:\n                response = raw_response.parse()\n            except json.JSONDecodeError as json_error:\n                raise AzureOpenAIError(\n                    status_code=raw_response.status_code or 500,\n                    message=f\"Failed to parse raw Azure embedding response: {json_error}\",\n                ) from json_error\n            if isinstance(response, str):\n                raise AzureOpenAIError(\n                    status_code=raw_response.status_code or 500,\n                    message=f\"Unexpected string response from Azure: {response[:500]}\",\n                )\n            stringified_response: Final = response.model_dump()\n\n            ## LOGGING\n            logging_obj.post_call(\n                input=input,\n                api_key=api_key,\n                additional_args={\"complete_input_dict\": data},\n                original_response=stringified_response,\n            )\n            embedding_response: Final = convert_to_model_response_object(","sourceCodeStart":680,"sourceCodeEnd":716,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/azure/azure.py#L680-L716","documentation":"When Azure's embedding endpoint returns a body that is not valid JSON, httpx's parse() raises JSONDecodeError, which LiteLLM converts to AzureOpenAIError carrying the upstream status code. The conversion exists deliberately (per the inline comment) so the router sees a status_code for cooldown logic and so httpx connection cleanup runs, preventing connection leaks under load.","triggerScenarios":"Azure or an intermediary (gateway, API management layer, corporate proxy) returns HTML error pages, empty bodies, or truncated JSON for an embeddings request; intermittent 5xx with non-JSON payloads.","commonSituations":"Traffic routed through Azure API Management with custom error templates; proxy timeouts returning HTML; partial responses under network stress.","solutions":["Capture the raw body (enable litellm debug/logging) to see what non-JSON payload came back.","If a gateway/proxy sits in front of Azure, exempt or fix its error pages for the embeddings path.","Retry with backoff — transient malformed responses usually succeed on retry; litellm router cooldown will mark the deployment and reroute.","If persistent, compare direct curl calls to the endpoint to confirm Azure itself is not misbehaving."],"exampleFix":"# before\nresp = await litellm.aembedding(model='azure/text-embedding-3-large', input=texts)  # raises AzureOpenAIError on bad JSON\n\n# after\nfrom litellm.exceptions import APIError\ntry:\n    resp = await litellm.aembedding(model='azure/text-embedding-3-large', input=texts)\nexcept APIError as e:\n    if 'Failed to parse raw Azure embedding response' in str(e):\n        await asyncio.sleep(2)\n        resp = await litellm.aembedding(model='azure/text-embedding-3-large', input=texts)\n    else:\n        raise","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"from litellm.exceptions import APIError\n\nasync def embed_with_retry(texts, attempts=3):\n    for i in range(attempts):\n        try:\n            return await litellm.aembedding(model='azure/text-embedding-3-large', input=texts)\n        except APIError as e:\n            if 'Failed to parse raw Azure embedding response' not in str(e) or i == attempts - 1:\n                raise\n            await asyncio.sleep(2 ** i)","preventionTips":["Enable litellm verbose logging in staging so raw non-JSON bodies are captured when this fires.","Prefer litellm.Router for embeddings so cooldown/reroute handles malformed responses automatically.","Keep gateway error pages out of the Azure embeddings route."],"tags":["azure","embeddings","json","network","retry","connection-leak"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}