{"record":{"id":"fcc93574502d306d","repo":"BerriAI/litellm","slug":"api-base-must-be-provided-for-hosted-vllm-rerank","errorCode":null,"errorMessage":"api_base must be provided for Hosted VLLM rerank","messagePattern":"api_base must be provided for Hosted VLLM rerank","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/hosted_vllm/rerank/transformation.py","lineNumber":55,"sourceCode":"    def __init__(self) -> None:\n        pass\n\n    def get_complete_url(\n        self,\n        api_base: str | None,\n        model: str,\n        optional_params: dict | None = None,\n    ) -> str:\n        if api_base:\n            # Remove trailing slashes and ensure clean base URL\n            api_base = api_base.rstrip(\"/\")\n            # Preserve backward compatibility\n            if api_base.endswith(\"/v1/rerank\"):\n                api_base = api_base.replace(\"/v1/rerank\", \"/rerank\")\n            elif not api_base.endswith(\"/rerank\"):\n                api_base = f\"{api_base}/rerank\"\n            return api_base\n        raise ValueError(\"api_base must be provided for Hosted VLLM rerank\")\n\n    def get_supported_cohere_rerank_params(self, model: str) -> list:\n        return [\n            \"query\",\n            \"documents\",\n            \"top_n\",\n            \"rank_fields\",\n            \"return_documents\",\n            \"instruction\",\n        ]\n\n    def map_cohere_rerank_params(\n        self,\n        non_default_params: dict | None,\n        model: str,\n        drop_params: bool,\n        query: str,\n        documents: list[str | dict[str, Any]],","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/hosted_vllm/rerank/transformation.py#L37-L73","documentation":"Raised by HostedVLLM RerankConfig.get_complete_url when api_base is falsy. Hosted vLLM rerank endpoints are self-hosted and have no default URL, so LiteLLM requires you to supply one (it would otherwise have nowhere to send the /rerank request). Note this handler does NOT fall back to the HOSTED_VLLM_API_BASE env var — only the explicit api_base argument is checked.","triggerScenarios":"Calling litellm.rerank(model='hosted_vllm/<reranker>', query=..., documents=[...]) without api_base. Setting only the HOSTED_VLLM_API_BASE env var does not help here, unlike the embeddings handler.","commonSituations":"Assuming the env var used for hosted_vllm chat/embeddings also covers rerank; migrating from Cohere rerank where no base URL was needed; reranker deployed at a different host than the LLM and the second URL was never wired up.","solutions":["Pass api_base to the rerank call: litellm.rerank(model='hosted_vllm/<model>', query=q, documents=docs, api_base='http://vllm-host:8000').","Point it at the vLLM rerank service; the code normalizes the path (accepts base, .../rerank, or legacy .../v1/rerank) — the bare base URL is safest.","If rerank should honor the env var in your setup, wrap the call site to read it yourself (os.environ.get('HOSTED_VLLM_API_BASE')) and pass it as api_base."],"exampleFix":"# before\nlitellm.rerank(model='hosted_vllm/bge-reranker-v2-m3', query='query', documents=docs)\n# raises ValueError: api_base must be provided for Hosted VLLM rerank\n\n# after\nlitellm.rerank(\n    model='hosted_vllm/bge-reranker-v2-m3',\n    query='query',\n    documents=docs,\n    api_base='http://localhost:8000',\n)\n","handlingStrategy":"validation","validationCode":"def vllm_rerank_base(api_base: str | None) -> str:\n    if not api_base:\n        raise ValueError(\"hosted_vllm rerank requires an explicit api_base (env var is NOT consulted)\")\n    return api_base","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Remember hosted_vllm rerank ignores HOSTED_VLLM_API_BASE — always pass api_base explicitly at the rerank call site.","Centralize your vLLM URLs in config and inject them into every hosted_vllm call (chat, embeddings, rerank, transcription) explicitly."],"tags":["hosted-vllm","rerank","api-base","configuration"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}