{"record":{"id":"cbb5ca8ccdbc02bb","repo":"BerriAI/litellm","slug":"vllmexception-original-exception-message","errorCode":null,"errorMessage":"VLLMException - {original_exception.message}","messagePattern":"VLLMException - (.+?)","errorType":"exception","errorClass":"APIConnectionError","httpStatus":null,"severity":"critical","filePath":"litellm/litellm_core_utils/exception_mapping_utils.py","lineNumber":1849,"sourceCode":"            message=f\"OllamaException: {original_exception}\",\n            llm_provider=\"ollama\",\n            model=model,\n        )\n\n\ndef _map_vllm_exception(\n    *,\n    model: str,\n    original_exception: _ProviderHTTPException,\n    custom_llm_provider: str,\n    error_str: str,\n    exception_type: str,\n    exception_provider: str,\n    extra_information: str,\n) -> None:\n    if hasattr(original_exception, \"status_code\"):\n        if original_exception.status_code == 0:\n            raise APIConnectionError(\n                message=f\"VLLMException - {original_exception.message}\",\n                llm_provider=\"vllm\",\n                model=model,\n                request=getattr(original_exception, \"request\", None),\n            )\n\n\ndef _map_azure_exception(\n    *,\n    model: str,\n    original_exception: _ProviderHTTPException,\n    custom_llm_provider: str,\n    error_str: str,\n    exception_type: str,\n    exception_provider: str,\n    extra_information: str,\n) -> None:\n    message = get_error_message(error_obj=original_exception)","sourceCodeStart":1831,"sourceCodeEnd":1867,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/litellm_core_utils/exception_mapping_utils.py#L1831-L1867","documentation":"litellm maps a vLLM error with status_code == 0 to litellm.APIConnectionError. Status 0 means no HTTP response at all - the TCP connection to the vLLM server failed (refused, reset, or DNS failure).","triggerScenarios":"Calling model='vllm/...' (or openai/ against a vLLM base) when the vLLM server process is down, the api_base URL/port is wrong, the pod crashed (OOM during model load), or a network policy blocks the connection. The client raises a connection-level exception with status_code 0 and this mapper converts it.","commonSituations":"vLLM workers OOM-killed while loading large models, k8s service endpoints not ready, wrong OPENAI_API_BASE in env, or healthcheck-less deployments routing to dead replicas.","solutions":["Verify the vLLM server process/container is alive and the port matches api_base","Health-check the endpoint: curl http://<host>:<port>/health (vLLM) or /v1/models","Fix the base URL passed to litellm (api_base= including http:// and port)","If OOM: raise memory limits or reduce gpu_memory_utilization / tensor_parallel size","Add readiness probes so traffic only routes to ready vLLM replicas"],"exampleFix":"# before\nlitellm.completion(model='vllm/meta-llama/Llama-3-8B', messages=msgs,\n                    api_base='http://localhost:8000')  # server crashed\n# after\nimport requests\nbase = 'http://localhost:8000'\nrequests.get(f'{base}/health', timeout=5).raise_for_status()  # gate the call\nlitellm.completion(model='vllm/meta-llama/Llama-3-8B', messages=msgs, api_base=base)","handlingStrategy":"validation","validationCode":"import requests\n\ndef vllm_ready(base: str) -> bool:\n    try:\n        return requests.get(f'{base}/health', timeout=3).status_code == 200\n    except requests.RequestException:\n        return False","typeGuard":"import litellm\n\ndef is_vllm_conn_error(e: Exception) -> bool:\n    return isinstance(e, litellm.APIConnectionError) and 'VLLMException' in str(e)","tryCatchPattern":"if not vllm_ready(base):\n    raise RuntimeError('vLLM not ready')\ntry:\n    litellm.completion(...)\nexcept litellm.APIConnectionError:\n    restart_or_failover()","preventionTips":["Run vLLM under a supervisor with restart policy","Set memory requests/limits so the model never OOMs at load","Health-check /health before sending user traffic"],"tags":["vllm","connection","self-hosted","litellm"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}