{"record":{"id":"e71892350bd05402","repo":"BerriAI/litellm","slug":"api-base-is-required-e71892","errorCode":null,"errorMessage":"api_base is required","messagePattern":"api_base is required","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/triton/completion/transformation.py","lineNumber":79,"sourceCode":"        model: str,\n        drop_params: bool,\n    ) -> dict:\n        for param, value in non_default_params.items():\n            if param == \"max_tokens\" or param == \"max_completion_tokens\":\n                optional_params[param] = value\n        return optional_params\n\n    def get_complete_url(\n        self,\n        api_base: str | None,\n        api_key: str | None,\n        model: str,\n        optional_params: dict,\n        litellm_params: dict,\n        stream: bool | None = None,\n    ) -> str:\n        if api_base is None:\n            raise ValueError(\"api_base is required\")\n        llm_type: Final = self._get_triton_llm_type(api_base)\n        if llm_type == \"generate\" and stream:\n            return api_base + \"_stream\"\n        return api_base\n\n    def transform_response(\n        self,\n        model: str,\n        raw_response: Response,\n        model_response: ModelResponse,\n        logging_obj: LiteLLMLoggingObj,\n        request_data: dict,\n        messages: list[AllMessageValues],\n        optional_params: dict,\n        litellm_params: dict,\n        encoding: Any,\n        api_key: str | None = None,\n        json_mode: bool | None = None,","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/triton/completion/transformation.py#L61-L97","documentation":"Triton is a self-hosted inference server, so LiteLLM has no default URL for it. get_complete_url requires an explicit api_base (typically the full gRPC-proxy/HTTP endpoint of your Triton model); when it is None this ValueError is raised before any request is built. The value should point at your Triton HTTP endpoint, e.g. http://triton:8000/v2/models/<model>/generate or .../infer.","triggerScenarios":"Calling completion(model=\"triton/...\", ...) or configuring a Router deployment for a triton/ model without api_base in litellm_params; relying on an api_base environment variable that isn't set; model_info entries copied from another provider that never included api_base.","commonSituations":"Self-hosting Triton (KServe, Triton Inference Server on K8s) and forgetting the endpoint in the deployment config; switching from OpenAI-style providers where the base URL is baked in; DNS/service names changing after a migration so configs were edited and api_base dropped.","solutions":["Pass api_base explicitly: completion(model=\"triton/llama\", api_base=\"http://triton-host:8000/v2/models/llama/generate\", ...).","In Router/proxy config, set litellm_params.api_base on the triton model_list entry.","Confirm the endpoint includes the model path and ends with /generate (or /infer) to also pass the llm-type check.","Health-check the URL (curl) to ensure the Triton HTTP server is reachable from the litellm process."],"exampleFix":"# before\nresp = litellm.completion(\n    model=\"triton/my-llm\",\n    messages=[{\"role\": \"user\", \"content\": \"hi\"}],\n)\n# -> ValueError: api_base is required\n\n# after\nresp = litellm.completion(\n    model=\"triton/my-llm\",\n    messages=[{\"role\": \"user\", \"content\": \"hi\"}],\n    api_base=\"http://triton:8000/v2/models/my-llm/generate\",\n)","handlingStrategy":"validation","validationCode":"def triton_endpoint_ready(api_base: str | None, model: str) -> bool:\n    \"\"\"True when a usable Triton HTTP endpoint is configured.\"\"\"\n    if not api_base:\n        return False\n    return api_base.endswith(\"/generate\") or api_base.endswith(\"/infer\")\n\n\nassert triton_endpoint_ready(deployment.api_base, deployment.model), \\\n    \"set api_base like http://host:8000/v2/models/<model>/generate\"","typeGuard":null,"tryCatchPattern":"try:\n    resp = litellm.completion(model=\"triton/my-llm\", messages=msgs)\nexcept ValueError as e:\n    if str(e) == \"api_base is required\":\n        raise RuntimeError(\n            \"triton deployment missing api_base (e.g. http://host:8000/v2/models/m/generate)\"\n        ) from e\n    raise","preventionTips":["Make api_base a required field in your deployment config schema for triton models (fail at config load).","Store full per-model Triton URLs (host, port 8000, model path, /generate or /infer) — never the bare server root.","Add a config linter that rejects triton model entries without a valid endpoint suffix."],"tags":["triton","self-hosted","api-base","configuration","litellm"],"backgroundTag":"missing-endpoint-url","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}