BerriAI/litellm · error · ValueError

query is required for Nvidia NIM rerank

Error message

query is required for Nvidia NIM rerank

What it means

Raised by litellm's Nvidia NIM rerank transformer in transform_rerank_request when 'query' is absent from optional_rerank_params. The native NIM /v1/ranking endpoint requires a query; litellm maps query to {text: query} and refuses to build a request without it.

Source

Thrown at litellm/llms/nvidia_nim/rerank/transformation.py:198

        model: str,
        optional_rerank_params: dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        """
        Transform request to Nvidia NIM format.

        Nvidia NIM expects:
        - query as {text: "..."}
        - documents as passages: [{text: "..."}, ...]
        - Optional: truncate (NONE or END), top_k

        Note: optional_rerank_params may contain provider-specific params like 'top_k' and 'truncate'
        that aren't in the OptionalRerankParams TypedDict but are passed through at runtime.
        The mapping from Cohere's 'top_n' to Nvidia's 'top_k' already happened in map_cohere_rerank_params.
        """
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Nvidia NIM rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Nvidia NIM rerank")

        query: Final = optional_rerank_params["query"]
        documents: Final = optional_rerank_params["documents"]

        # Transform query to object format
        query_obj: Final[NvidiaNimQueryObject] = {"text": query}

        # Transform documents to passages format
        passages: Final[list[NvidiaNimPassageObject]] = []
        for doc in documents:
            if isinstance(doc, str):
                passages.append({"text": doc})
            elif isinstance(doc, dict):
                # Preserve only the structured passage fields supported by the
                # selected rerank route.
                supported_fields: NvidiaNimPassageObject = {}  # mutable-ok: assembling a request TypedDict

View on GitHub (pinned to 6c2dcb801b)

Solutions

  1. Always pass query explicitly: litellm.rerank(model=..., query='...', documents=[...]).
  2. Check your kwargs builder includes the literal key 'query'.
  3. Validate required keys before calling (see defense code).

Example fix

# before
litellm.rerank(model="nvidia_nim/nv-rerankqa-mistral-4b-v3", documents=docs, q="what is litellm?")

# after
litellm.rerank(model="nvidia_nim/nv-rerankqa-mistral-4b-v3", documents=docs, query="what is litellm?")
Defensive patterns

Strategy: type-guard

Validate before calling

def validate_rerank_args(query: str | None, documents: list | None) -> None:
    if not query or not query.strip():
        raise ValueError("query is required for rerank")
    if not documents:
        raise ValueError("documents is required for rerank")

Type guard

from typing import Any, TypeGuard

def is_rerank_ready(documents: Any) -> TypeGuard[list[str]]:
    return isinstance(documents, list) and len(documents) > 0 and all(isinstance(d, str) and d.strip() for d in documents)

Try / catch

try:
    litellm.rerank(model=m, query=q, documents=docs)
except ValueError as e:
    if "query is required" in str(e):
        raise HTTPException(400, "missing query") from e
    raise

Prevention

When it happens

Trigger: Calling litellm.rerank(model='nvidia_nim/...') without the query argument, or passing it under a different name (e.g. q=, search_query=) so it never reaches optional_rerank_params.

Common situations: Adapting code from another rerank API with different parameter names, or building kwargs dynamically and accidentally dropping 'query'.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/1ac212e47ca5868b. Report an issue: GitHub.