BerriAI/litellm · error · ValueError

query is required for Vertex AI rerank

Error message

query is required for Vertex AI rerank

What it means

ValueError raised in transform_rerank_request when 'query' is absent from the rerank parameters. LiteLLM exposes rerank in the Cohere format where both query and documents are mandatory; the Vertex AI Discovery Engine ranking API has no default query, so the request cannot be transformed.

Source

Thrown at litellm/llms/vertex_ai/rerank/transformation.py:117

        # If 'Authorization' is provided in headers, it overrides the default.
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        """
        Transform the request from Cohere format to Vertex AI Discovery Engine format
        """
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Vertex AI rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Vertex AI rerank")

        query: Final = optional_rerank_params["query"]
        documents: Final = optional_rerank_params["documents"]
        top_n: Final = optional_rerank_params.get("top_n", None)
        return_documents: Final = optional_rerank_params.get("return_documents", True)

        # Convert documents to records format
        records: Final = []
        for idx, document in enumerate(documents):
            if isinstance(document, str):
                content = document
                title = " ".join(document.split()[:3])  # First 3 words as title
            else:
                # Handle dict format
                content = document.get("text", str(document))
                title = document.get("title", " ".join(content.split()[:3]))

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Pass query='...' to litellm.rerank
2. Check for renamed keys like q or search_query in your params dict
3. Reject empty queries upstream instead of forwarding them

Example fix

# before
litellm.rerank(model='vertex_ai/semantic-ranker', documents=docs)

# after
litellm.rerank(model='vertex_ai/semantic-ranker', query='what is litellm?', documents=docs)

def valid_rerank_params(params: dict) -> bool:
    return bool(params.get('query'))

def is_rerank_request(value: dict) -> bool:
    return isinstance(value, dict) and isinstance(value.get('query'), str) and bool(value['query'].strip())

try:
    resp = litellm.rerank(model=model, **params)
except ValueError as e:
    if 'query is required' in str(e):
        raise SystemExit('Missing rerank query')
    raise

When it happens

Trigger: Calling litellm.rerank with a vertex_ai model without a query argument, or with the query renamed/nested (e.g. 'q', 'text', or tucked inside another dict).

Common situations: Migrating from search APIs with different parameter names; building params dynamically and skipping empty queries; schema confusion between rerank and retrieval APIs.

Understand the failure class

Background: Missing required parameter errors: what 'X is required' and 'the required X param is missing' mean, and how to fix them — this error's family across 27 libraries.

Related errors

• documents is required for Vertex AI rerank
• Gemini image edit requires at least one image.
• Either 'model' or 'agent' must be provided
• query is required for Hosted VLLM rerank
• documents is required for Hosted VLLM rerank

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/baaaa48a77303f7d. Report an issue: GitHub.