BerriAI/litellm · error · ValueError

documents is required for Hosted VLLM rerank

Error message

documents is required for Hosted VLLM rerank

What it means

Raised by HostedVLLM RerankConfig.transform_rerank_request when the optional_rerank_params dict lacks a 'documents' key. The vLLM rerank endpoint needs the list of documents to score, so LiteLLM refuses to build the request payload without it.

Source

Thrown at litellm/llms/hosted_vllm/rerank/transformation.py:137

        # If 'Authorization' is provided in headers, it overrides the default.
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Hosted VLLM rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Hosted VLLM rerank")

        rerank_request: Final = RerankRequest(
            model=model,
            query=optional_rerank_params["query"],
            documents=optional_rerank_params["documents"],
            top_n=optional_rerank_params.get("top_n", None),
            rank_fields=optional_rerank_params.get("rank_fields", None),
            return_documents=optional_rerank_params.get("return_documents", None),
            instruction=optional_rerank_params.get("instruction", None),
        )
        return rerank_request.model_dump(exclude_none=True)

    def transform_rerank_response(
        self,
        model: str,
        raw_response: httpx.Response,
        model_response: RerankResponse,
        logging_obj: LiteLLMLoggingObj,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Always pass documents: litellm.rerank(model='hosted_vllm/...', query=q, documents=docs, api_base=...).
2. Short-circuit in RAG flows: skip rerank entirely when retrieval returns no documents instead of calling it without the key.
3. Validate dynamic param dicts before the call: required = {'query','documents'}; missing = required - params.keys().

Example fix

# before
litellm.rerank(model='hosted_vllm/bge-reranker-v2-m3', query=q, api_base=base)
# raises ValueError: documents is required for Hosted VLLM rerank

# after
if retrieved_docs:
    litellm.rerank(model='hosted_vllm/bge-reranker-v2-m3', query=q,
                   documents=retrieved_docs, api_base=base)

def validate_rerank_request(params: dict) -> None:
    for key in ("query", "documents"):
        if key not in params:
            raise ValueError(f"'{key}' is required for hosted_vllm rerank")

def maybe_rerank(params: dict) -> dict | None:
    if not params.get("documents"):
        return None  # skip rerank when retrieval was empty
    validate_rerank_request(params)
    return params

When it happens

Trigger: Calling litellm.rerank(model='hosted_vllm/...', query='...') with no documents; forwarding a dynamically built params dict where documents failed to populate (empty retrieval results path that still calls rerank, key named 'docs' instead of 'documents').

Common situations: RAG pipeline where the retrieval step returned nothing and the rerank step still runs; renaming variables during refactor (docs vs documents); wrapper that conditionally includes documents only when non-empty (note: an empty list [] would pass this check but fail server-side — omitting the key is what raises).

Related errors

• query is required for Hosted VLLM rerank
• Hosted VLLM does not support max_chunks_per_doc
• No results found in the response={response}
• Missing required fields in the result={result}
• No results found in the response={response}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/b7d8a11ca873fef6. Report an issue: GitHub.