BerriAI/litellm · error · ValueError

query is required for Hosted VLLM rerank

Error message

query is required for Hosted VLLM rerank

What it means

Raised by HostedVLLM RerankConfig.transform_rerank_request when the optional_rerank_params dict passed to the rerank call does not contain a 'query' key. The vLLM rerank endpoint requires a query to score documents against, so LiteLLM fails fast before building the RerankRequest payload.

Source

Thrown at litellm/llms/hosted_vllm/rerank/transformation.py:135

            "content-type": "application/json",
        }

        # If 'Authorization' is provided in headers, it overrides the default.
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Hosted VLLM rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Hosted VLLM rerank")

        rerank_request: Final = RerankRequest(
            model=model,
            query=optional_rerank_params["query"],
            documents=optional_rerank_params["documents"],
            top_n=optional_rerank_params.get("top_n", None),
            rank_fields=optional_rerank_params.get("rank_fields", None),
            return_documents=optional_rerank_params.get("return_documents", None),
            instruction=optional_rerank_params.get("instruction", None),
        )
        return rerank_request.model_dump(exclude_none=True)

    def transform_rerank_response(
        self,
        model: str,
        raw_response: httpx.Response,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Always pass query: litellm.rerank(model='hosted_vllm/...', query='...', documents=[...], api_base=...).
2. When building params dynamically, validate required keys before the call: assert {'query','documents'} <= params.keys().
3. If the query is genuinely absent in your flow, you need one — rerank scores documents relative to a query, so design the caller to always supply it.

Example fix

# before
params = {'documents': docs}  # query missing
litellm.rerank(model='hosted_vllm/bge-reranker-v2-m3', api_base=base, **params)
# raises ValueError: query is required for Hosted VLLM rerank

# after
params = {'query': user_question, 'documents': docs}
litellm.rerank(model='hosted_vllm/bge-reranker-v2-m3', api_base=base, **params)

REQUIRED_RERANK_KEYS = {"query", "documents"}

def validate_rerank_params(params: dict) -> None:
    missing = REQUIRED_RERANK_KEYS - params.keys()
    if missing:
        raise ValueError(f"rerank call missing required keys: {sorted(missing)}")

When it happens

Trigger: Calling litellm.rerank(model='hosted_vllm/...', documents=docs) without query=; or constructing the call dynamically from a dict that may omit 'query' (e.g. **params where params lacks the key). Note litellm.rerank's own signature normally enforces query, so this is most often hit via programmatic param dicts or wrapper layers.

Common situations: Generic retrieval wrapper that forwards an optional params dict; refactoring left query out of a call site; default/fallback code path that reranks 'documents only' (not valid for vLLM rerank).

Related errors

• documents is required for Hosted VLLM rerank
• Hosted VLLM does not support max_chunks_per_doc
• No results found in the response={response}
• Missing required fields in the result={result}
• No results found in the response={response}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/1177b6f9d5296b10. Report an issue: GitHub.