BerriAI/litellm · error · ValueError

query is required for Hosted VLLM rerank

Error message

query is required for Hosted VLLM rerank

What it means

Before sending a rerank request to vLLM, transform_rerank_request() validates the Cohere-style payload and requires a top-level 'query' string. vLLM's /rerank endpoint (Jina-compatible) cannot rank without a query, so its absence raises ValueError before any network call. The same function also requires 'documents'.

Solutions

1. Pass a non-empty query string: litellm.rerank(model=..., api_base=..., query='what is litellm', documents=[...])
2. If building params dynamically, assert 'query' in params before calling litellm.rerank
3. Check for upstream code that pops or filters the query key
4. Use litellm.validate_rerank_params or inspect RerankRequest fields if unsure of the expected shape

Example fix

# before
litellm.rerank(model="hosted_vllm/jina-reranker", api_base=base, documents=["a", "b"])

# after
litellm.rerank(model="hosted_vllm/jina-reranker", api_base=base, query="find a", documents=["a", "b"])

if not (query and query.strip()):
    raise ValueError("rerank requires a non-empty query")

When it happens

Trigger: litellm.rerank(model='hosted_vllm/...', documents=[...]) with no query kwarg; building optional_rerank_params dynamically and the query key getting dropped; passing query=None explicitly (the key exists upstream but is stripped before this point).

Common situations: Typos in the kwarg name (q=, search_query=); refactoring from a provider whose API makes the query optional; a wrapper that filters out falsy values and removes query='' before delegating.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• documents is required for Hosted VLLM rerank
• api_base must be provided for Hosted VLLM rerank
• Error parsing response
• file_id or file_data is required
• Hosted VLLM does not support max_chunks_per_doc

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/1177b6f9d5296b10. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/hosted_vllm/rerank/transformation.py:135

            "content-type": "application/json",
        }

        # If 'Authorization' is provided in headers, it overrides the default.
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Hosted VLLM rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Hosted VLLM rerank")

        rerank_request: Final = RerankRequest(
            model=model,
            query=optional_rerank_params["query"],
            documents=optional_rerank_params["documents"],
            top_n=optional_rerank_params.get("top_n", None),
            rank_fields=optional_rerank_params.get("rank_fields", None),
            return_documents=optional_rerank_params.get("return_documents", None),
            instruction=optional_rerank_params.get("instruction", None),
        )
        return rerank_request.model_dump(exclude_none=True)

    def transform_rerank_response(
        self,
        model: str,
        raw_response: httpx.Response,

View on GitHub (pinned to 77b7c6c40c)