What causes "api_base must be provided for Hosted VLLM rerank" in BerriAI/litellm?

Pass api_base to the rerank call: litellm.rerank(model='hosted_vllm/ ', query=q, documents=docs, api_base='http://vllm-host:8000').

BerriAI/litellm · error · ValueError

api_base must be provided for Hosted VLLM rerank

Error message

api_base must be provided for Hosted VLLM rerank

What it means

Raised by HostedVLLM RerankConfig.get_complete_url when api_base is falsy. Hosted vLLM rerank endpoints are self-hosted and have no default URL, so LiteLLM requires you to supply one (it would otherwise have nowhere to send the /rerank request). Note this handler does NOT fall back to the HOSTED_VLLM_API_BASE env var — only the explicit api_base argument is checked.

Source

Thrown at litellm/llms/hosted_vllm/rerank/transformation.py:55

    def __init__(self) -> None:
        pass

    def get_complete_url(
        self,
        api_base: str | None,
        model: str,
        optional_params: dict | None = None,
    ) -> str:
        if api_base:
            # Remove trailing slashes and ensure clean base URL
            api_base = api_base.rstrip("/")
            # Preserve backward compatibility
            if api_base.endswith("/v1/rerank"):
                api_base = api_base.replace("/v1/rerank", "/rerank")
            elif not api_base.endswith("/rerank"):
                api_base = f"{api_base}/rerank"
            return api_base
        raise ValueError("api_base must be provided for Hosted VLLM rerank")

    def get_supported_cohere_rerank_params(self, model: str) -> list:
        return [
            "query",
            "documents",
            "top_n",
            "rank_fields",
            "return_documents",
            "instruction",
        ]

    def map_cohere_rerank_params(
        self,
        non_default_params: dict | None,
        model: str,
        drop_params: bool,
        query: str,
        documents: list[str | dict[str, Any]],

View on GitHub (pinned to 6c2dcb801b)

Solutions

Pass api_base to the rerank call: litellm.rerank(model='hosted_vllm/<model>', query=q, documents=docs, api_base='http://vllm-host:8000').
Point it at the vLLM rerank service; the code normalizes the path (accepts base, .../rerank, or legacy .../v1/rerank) — the bare base URL is safest.
If rerank should honor the env var in your setup, wrap the call site to read it yourself (os.environ.get('HOSTED_VLLM_API_BASE')) and pass it as api_base.

Example fix

# before
litellm.rerank(model='hosted_vllm/bge-reranker-v2-m3', query='query', documents=docs)
# raises ValueError: api_base must be provided for Hosted VLLM rerank

# after
litellm.rerank(
    model='hosted_vllm/bge-reranker-v2-m3',
    query='query',
    documents=docs,
    api_base='http://localhost:8000',
)

Defensive patterns

Strategy: validation

Validate before calling

def vllm_rerank_base(api_base: str | None) -> str:
    if not api_base:
        raise ValueError("hosted_vllm rerank requires an explicit api_base (env var is NOT consulted)")
    return api_base

Prevention

Remember hosted_vllm rerank ignores HOSTED_VLLM_API_BASE — always pass api_base explicitly at the rerank call site.
Centralize your vLLM URLs in config and inject them into every hosted_vllm call (chat, embeddings, rerank, transcription) explicitly.

When it happens

Trigger: Calling litellm.rerank(model='hosted_vllm/<reranker>', query=..., documents=[...]) without api_base. Setting only the HOSTED_VLLM_API_BASE env var does not help here, unlike the embeddings handler.

Common situations: Assuming the env var used for hosted_vllm chat/embeddings also covers rerank; migrating from Cohere rerank where no base URL was needed; reranker deployed at a different host than the LLM and the second URL was never wired up.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/fcc93574502d306d. Report an issue: GitHub.