BerriAI/litellm · error · ValueError

api_base must be provided for Hosted VLLM rerank. Set in cal

Error message

api_base must be provided for Hosted VLLM rerank. Set in call or via HOSTED_VLLM_API_BASE env var.

What it means

API-base resolution guard for Hosted VLLM rerank: hosted VLLM has no default rerank URL, so without an api_base in the call or the HOSTED_VLLM_API_BASE env var the endpoint is unknown and the call is rejected.

Source

Thrown at litellm/rerank_api/main.py:375

                rank_fields=rank_fields,
                return_documents=return_documents,
                max_chunks_per_doc=max_chunks_per_doc,
                _is_async=_is_async,
                optional_params=optional_params.model_dump(exclude_unset=True),
                timeout=optional_params.timeout,
                api_base=api_base,
                extra_headers=merged_headers,
                logging_obj=litellm_logging_obj,
                client=client,
            )
        elif _custom_llm_provider == litellm.LlmProviders.HOSTED_VLLM:
            # Implement Hosted VLLM rerank logic
            api_key = dynamic_api_key or optional_params.api_key or get_secret_str("HOSTED_VLLM_API_KEY")

            api_base = dynamic_api_base or optional_params.api_base or get_secret_str("HOSTED_VLLM_API_BASE")

            if api_base is None:
                raise ValueError(
                    "api_base must be provided for Hosted VLLM rerank. Set in call or via HOSTED_VLLM_API_BASE env var."
                )

            response = base_llm_http_handler.rerank(
                model=model,
                custom_llm_provider=_custom_llm_provider,
                provider_config=rerank_provider_config,
                optional_rerank_params=optional_rerank_params,
                logging_obj=litellm_logging_obj,
                timeout=optional_params.timeout,
                api_key=api_key,
                api_base=api_base,
                _is_async=_is_async,
                headers=headers or litellm.headers or {},
                client=client,
                model_response=model_response,
                litellm_params=rerank_litellm_params,
            )

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Pass api_base in the call or set HOSTED_VLLM_API_BASE.

When it happens

Trigger: Thrown at litellm/rerank_api/main.py:375 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/1c5179227809340a. Report an issue: GitHub.