BerriAI/litellm · error · ValueError

api_base is required

Error message

api_base is required

What it means

Triton is a self-hosted inference server, so LiteLLM has no default URL for it. get_complete_url requires an explicit api_base (typically the full gRPC-proxy/HTTP endpoint of your Triton model); when it is None this ValueError is raised before any request is built. The value should point at your Triton HTTP endpoint, e.g. http://triton:8000/v2/models/<model>/generate or .../infer.

Source

Thrown at litellm/llms/triton/completion/transformation.py:79

        model: str,
        drop_params: bool,
    ) -> dict:
        for param, value in non_default_params.items():
            if param == "max_tokens" or param == "max_completion_tokens":
                optional_params[param] = value
        return optional_params

    def get_complete_url(
        self,
        api_base: str | None,
        api_key: str | None,
        model: str,
        optional_params: dict,
        litellm_params: dict,
        stream: bool | None = None,
    ) -> str:
        if api_base is None:
            raise ValueError("api_base is required")
        llm_type: Final = self._get_triton_llm_type(api_base)
        if llm_type == "generate" and stream:
            return api_base + "_stream"
        return api_base

    def transform_response(
        self,
        model: str,
        raw_response: Response,
        model_response: ModelResponse,
        logging_obj: LiteLLMLoggingObj,
        request_data: dict,
        messages: list[AllMessageValues],
        optional_params: dict,
        litellm_params: dict,
        encoding: Any,
        api_key: str | None = None,
        json_mode: bool | None = None,

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Pass api_base explicitly: completion(model="triton/llama", api_base="http://triton-host:8000/v2/models/llama/generate", ...).
2. In Router/proxy config, set litellm_params.api_base on the triton model_list entry.
3. Confirm the endpoint includes the model path and ends with /generate (or /infer) to also pass the llm-type check.
4. Health-check the URL (curl) to ensure the Triton HTTP server is reachable from the litellm process.

Example fix

# before
resp = litellm.completion(
    model="triton/my-llm",
    messages=[{"role": "user", "content": "hi"}],
)
# -> ValueError: api_base is required

# after
resp = litellm.completion(
    model="triton/my-llm",
    messages=[{"role": "user", "content": "hi"}],
    api_base="http://triton:8000/v2/models/my-llm/generate",
)

def triton_endpoint_ready(api_base: str | None, model: str) -> bool:
    """True when a usable Triton HTTP endpoint is configured."""
    if not api_base:
        return False
    return api_base.endswith("/generate") or api_base.endswith("/infer")


assert triton_endpoint_ready(deployment.api_base, deployment.model), \
    "set api_base like http://host:8000/v2/models/<model>/generate"

try:
    resp = litellm.completion(model="triton/my-llm", messages=msgs)
except ValueError as e:
    if str(e) == "api_base is required":
        raise RuntimeError(
            "triton deployment missing api_base (e.g. http://host:8000/v2/models/m/generate)"
        ) from e
    raise

When it happens

Trigger: Calling completion(model="triton/...", ...) or configuring a Router deployment for a triton/ model without api_base in litellm_params; relying on an api_base environment variable that isn't set; model_info entries copied from another provider that never included api_base.

Common situations: Self-hosting Triton (KServe, Triton Inference Server on K8s) and forgetting the endpoint in the deployment config; switching from OpenAI-style providers where the base URL is baked in; DNS/service names changing after a migration so configs were edited and api_base dropped.

Related errors

• Invalid Triton API base: {api_base}
• api_base is required for triton. Please pass `api_base`
• api base needs to be a string. api_base={api_base}
• No api base was set. Please provide an api_base, or set the
• api_base is required for hosted_vllm embeddings

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/e71892350bd05402. Report an issue: GitHub.