{"record":{"id":"fcc93574502d306d","repo":"BerriAI/litellm","slug":"api-base-must-be-provided-for-hosted-vllm-rerank","errorCode":null,"errorMessage":"api_base must be provided for Hosted VLLM rerank","messagePattern":"api_base must be provided for Hosted VLLM rerank","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/hosted_vllm/rerank/transformation.py","lineNumber":55,"sourceCode":"    def __init__(self) -> None:\n        pass\n\n    def get_complete_url(\n        self,\n        api_base: str | None,\n        model: str,\n        optional_params: dict | None = None,\n    ) -> str:\n        if api_base:\n            # Remove trailing slashes and ensure clean base URL\n            api_base = api_base.rstrip(\"/\")\n            # Preserve backward compatibility\n            if api_base.endswith(\"/v1/rerank\"):\n                api_base = api_base.replace(\"/v1/rerank\", \"/rerank\")\n            elif not api_base.endswith(\"/rerank\"):\n                api_base = f\"{api_base}/rerank\"\n            return api_base\n        raise ValueError(\"api_base must be provided for Hosted VLLM rerank\")\n\n    def get_supported_cohere_rerank_params(self, model: str) -> list:\n        return [\n            \"query\",\n            \"documents\",\n            \"top_n\",\n            \"rank_fields\",\n            \"return_documents\",\n            \"instruction\",\n        ]\n\n    def map_cohere_rerank_params(\n        self,\n        non_default_params: dict | None,\n        model: str,\n        drop_params: bool,\n        query: str,\n        documents: list[str | dict[str, Any]],","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/llms/hosted_vllm/rerank/transformation.py#L37-L73","documentation":"The HostedVLLM rerank transformation's get_complete_url() requires an explicit base URL for the same reason as other hosted_vllm endpoints: there is no default host. It normalizes the URL (strips trailing slashes, rewrites /v1/rerank to /rerank, appends /rerank if missing) and raises ValueError when api_base is falsy. This api_base comes from the call or the HOSTED_VLLM_API_BASE environment variable.","triggerScenarios":"litellm.rerank(model='hosted_vllm/<reranker>', query=..., documents=[...]) invoked without api_base=... and without HOSTED_VLLM_API_BASE set.","commonSituations":"Self-hosting a reranker (e.g. a Jina/Cohere-compatible rerank model) in vLLM and forgetting the base URL in one call path; env var set only in the dev shell, not in CI or the container; typo in the env var name.","solutions":["Pass api_base: litellm.rerank(model='hosted_vllm/jinaai/jina-reranker-v2-base-multilingual', api_base='http://localhost:8000', query=..., documents=[...])","Or export HOSTED_VLLM_API_BASE=http://localhost:8000 and confirm the process sees it","If you already include the full path, either /rerank or /v1/rerank endings are accepted and normalized","Verify the server actually exposes the rerank route (curl <api_base>/rerank) and was started with a rerank-capable model"],"exampleFix":"# before\nlitellm.rerank(model=\"hosted_vllm/jinaai/jina-reranker-v2-base-multilingual\", query=q, documents=docs)\n\n# after\nlitellm.rerank(\n    model=\"hosted_vllm/jinaai/jina-reranker-v2-base-multilingual\",\n    api_base=\"http://localhost:8000\",\n    query=q,\n    documents=docs,\n)","handlingStrategy":"validation","validationCode":"import os\n\nbase = api_base or os.getenv(\"HOSTED_VLLM_API_BASE\")\nassert base, \"hosted_vllm rerank requires api_base or HOSTED_VLLM_API_BASE\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Store reranker endpoint next to the model name in config so they deploy together","Distinguish the rerank server's base from the chat server's base in env naming (e.g. RERANK_API_BASE) and map it into the call","Verify /rerank is reachable after every infra change"],"tags":["hosted-vllm","rerank","api-base","configuration","litellm"],"backgroundTag":"missing-api-base","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}