BerriAI/litellm · error · ValueError

documents is required for Hosted VLLM rerank

Error message

documents is required for Hosted VLLM rerank

What it means

transform_rerank_request() also validates that the rerank payload contains 'documents' (the list of strings to be scored against the query). vLLM's /rerank endpoint requires both query and documents, so a missing documents key raises ValueError client-side before the HTTP request. The value is forwarded into a RerankRequest pydantic model alongside top_n, rank_fields, return_documents, and instruction.

Solutions

1. Pass the list to rank: litellm.rerank(model=..., api_base=..., query='...', documents=['doc1', 'doc2'])
2. Assert documents is a non-empty list before calling
3. Check wrappers for keys named docs/texts instead of documents
4. Return early in retrieval pipelines when the candidate set is empty rather than calling rerank

Example fix

# before
litellm.rerank(model="hosted_vllm/jina-reranker", api_base=base, query="find a")

# after
litellm.rerank(model="hosted_vllm/jina-reranker", api_base=base, query="find a", documents=["a", "b"])

if not documents:
    raise ValueError("rerank requires a non-empty documents list")

When it happens

Trigger: litellm.rerank(model='hosted_vllm/...', query='...') with no documents kwarg; passing an empty/None documents value that a wrapper strips before the call; confusing rerank (score existing docs) with completion generation.

Common situations: Refactoring retrieval code where documents were fetched conditionally and the list ended up absent; passing documents under a different kwarg (docs=, texts=); calling rerank as a similarity check without supplying the corpus.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• query is required for Hosted VLLM rerank
• api_base must be provided for Hosted VLLM rerank
• Error parsing response
• file_id or file_data is required
• Hosted VLLM does not support max_chunks_per_doc

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/b7d8a11ca873fef6. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/hosted_vllm/rerank/transformation.py:137

        # If 'Authorization' is provided in headers, it overrides the default.
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Hosted VLLM rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Hosted VLLM rerank")

        rerank_request: Final = RerankRequest(
            model=model,
            query=optional_rerank_params["query"],
            documents=optional_rerank_params["documents"],
            top_n=optional_rerank_params.get("top_n", None),
            rank_fields=optional_rerank_params.get("rank_fields", None),
            return_documents=optional_rerank_params.get("return_documents", None),
            instruction=optional_rerank_params.get("instruction", None),
        )
        return rerank_request.model_dump(exclude_none=True)

    def transform_rerank_response(
        self,
        model: str,
        raw_response: httpx.Response,
        model_response: RerankResponse,
        logging_obj: LiteLLMLoggingObj,

View on GitHub (pinned to 77b7c6c40c)