BerriAI/litellm · error · ValueError

api_base must be provided for Hosted VLLM rerank

Error message

api_base must be provided for Hosted VLLM rerank

What it means

The HostedVLLM rerank transformation's get_complete_url() requires an explicit base URL for the same reason as other hosted_vllm endpoints: there is no default host. It normalizes the URL (strips trailing slashes, rewrites /v1/rerank to /rerank, appends /rerank if missing) and raises ValueError when api_base is falsy. This api_base comes from the call or the HOSTED_VLLM_API_BASE environment variable.

Solutions

1. Pass api_base: litellm.rerank(model='hosted_vllm/jinaai/jina-reranker-v2-base-multilingual', api_base='http://localhost:8000', query=..., documents=[...])
2. Or export HOSTED_VLLM_API_BASE=http://localhost:8000 and confirm the process sees it
3. If you already include the full path, either /rerank or /v1/rerank endings are accepted and normalized
4. Verify the server actually exposes the rerank route (curl <api_base>/rerank) and was started with a rerank-capable model

Example fix

# before
litellm.rerank(model="hosted_vllm/jinaai/jina-reranker-v2-base-multilingual", query=q, documents=docs)

# after
litellm.rerank(
    model="hosted_vllm/jinaai/jina-reranker-v2-base-multilingual",
    api_base="http://localhost:8000",
    query=q,
    documents=docs,
)

import os

base = api_base or os.getenv("HOSTED_VLLM_API_BASE")
assert base, "hosted_vllm rerank requires api_base or HOSTED_VLLM_API_BASE"

When it happens

Trigger: litellm.rerank(model='hosted_vllm/<reranker>', query=..., documents=[...]) invoked without api_base=... and without HOSTED_VLLM_API_BASE set.

Common situations: Self-hosting a reranker (e.g. a Jina/Cohere-compatible rerank model) in vLLM and forgetting the base URL in one call path; env var set only in the dev shell, not in CI or the container; typo in the env var name.

Related errors

• api_base is required for hosted_vllm embeddings
• api_base not set for Hosted VLLM responses API. Set via…
• api_base is required
• api_base must be provided for Hosted VLLM rerank
• api base needs to be a string. api_base=

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/fcc93574502d306d. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/hosted_vllm/rerank/transformation.py:55

    def __init__(self) -> None:
        pass

    def get_complete_url(
        self,
        api_base: str | None,
        model: str,
        optional_params: dict | None = None,
    ) -> str:
        if api_base:
            # Remove trailing slashes and ensure clean base URL
            api_base = api_base.rstrip("/")
            # Preserve backward compatibility
            if api_base.endswith("/v1/rerank"):
                api_base = api_base.replace("/v1/rerank", "/rerank")
            elif not api_base.endswith("/rerank"):
                api_base = f"{api_base}/rerank"
            return api_base
        raise ValueError("api_base must be provided for Hosted VLLM rerank")

    def get_supported_cohere_rerank_params(self, model: str) -> list:
        return [
            "query",
            "documents",
            "top_n",
            "rank_fields",
            "return_documents",
            "instruction",
        ]

    def map_cohere_rerank_params(
        self,
        non_default_params: dict | None,
        model: str,
        drop_params: bool,
        query: str,
        documents: list[str | dict[str, Any]],

View on GitHub (pinned to 77b7c6c40c)