BerriAI/litellm · error · ValueError

api_base is required for Infinity rerank

Error message

api_base is required for Infinity rerank

What it means

Raised by the Infinity rerank config when api_base is None while constructing the /rerank URL. Infinity is self-hosted; litellm has no default endpoint for it, so the call is aborted with a ValueError before any HTTP request.

Source

Thrown at litellm/llms/infinity/rerank/transformation.py:36

    RerankResponse,
    RerankResponseDocument,
    RerankResponseMeta,
    RerankResponseResult,
    RerankTokens,
)

from ..common_utils import InfinityError


class InfinityRerankConfig(CohereRerankConfig):
    def get_complete_url(
        self,
        api_base: str | None,
        model: str,
        optional_params: dict | None = None,
    ) -> str:
        if api_base is None:
            raise ValueError("api_base is required for Infinity rerank")
        # Remove trailing slashes and ensure clean base URL
        api_base = api_base.rstrip("/")
        if not api_base.endswith("/rerank"):
            api_base = f"{api_base}/rerank"
        return api_base

    def validate_environment(
        self,
        headers: dict,
        model: str,
        api_key: str | None = None,
        optional_params: dict | None = None,
    ) -> dict:
        if api_key is None:
            api_key = get_secret_str("INFINITY_API_KEY") or get_secret_str("INFINITY_API_KEY") or litellm.infinity_key

        default_headers: Final = {
            "Authorization": f"Bearer {api_key}",

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Pass api_base explicitly: litellm.rerank(model='infinity/BAAI/bge-reranker-v2-m3', query=q, documents=docs, api_base='http://localhost:7111').
2. Add api_base to the model entry in the litellm proxy config.
3. Verify the Infinity instance exposes /rerank (recent Infinity versions) at that base.

Example fix

# before
litellm.rerank(model='infinity/BAAI/bge-reranker-v2-m3', query=q, documents=docs)

# after
litellm.rerank(model='infinity/BAAI/bge-reranker-v2-m3', query=q, documents=docs, api_base='http://localhost:7111')

def resolve_infinity_rerank_base(cfg: dict) -> str:
    base = cfg.get('api_base') or os.environ.get('INFINITY_API_BASE')
    if not base:
        raise ValueError('api_base required for infinity rerank; set INFINITY_API_BASE')
    return base

When it happens

Trigger: litellm.rerank(model='infinity/<model>', query=..., documents=...) with api_base omitted (neither passed nor present in proxy model config).

Common situations: Proxy config entry for the infinity reranker missing the api_base field; migrating from a cloud reranker and forgetting the self-host URL.

Related errors

• api_base is required for Infinity embeddings
• api_base must be provided for Hosted VLLM rerank
• {raw_response.text}
• API base is required for OpenAI image variations
• API base is required for Topaz image variations

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/4da4089c19bee60b. Report an issue: GitHub.