What causes "documents is required for Nvidia NIM rerank" in BerriAI/litellm?

Pass documents=[...] alongside query.

BerriAI/litellm · error · ValueError

documents is required for Nvidia NIM rerank

Error message

documents is required for Nvidia NIM rerank

What it means

Raised by litellm's Nvidia NIM rerank transformer in transform_rerank_request when 'documents' is absent from optional_rerank_params. The native NIM ranking endpoint requires passages to rank; without documents there is nothing to send, so litellm aborts request construction.

Source

Thrown at litellm/llms/nvidia_nim/rerank/transformation.py:200

        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        """
        Transform request to Nvidia NIM format.

        Nvidia NIM expects:
        - query as {text: "..."}
        - documents as passages: [{text: "..."}, ...]
        - Optional: truncate (NONE or END), top_k

        Note: optional_rerank_params may contain provider-specific params like 'top_k' and 'truncate'
        that aren't in the OptionalRerankParams TypedDict but are passed through at runtime.
        The mapping from Cohere's 'top_n' to Nvidia's 'top_k' already happened in map_cohere_rerank_params.
        """
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for Nvidia NIM rerank")
        if "documents" not in optional_rerank_params:
            raise ValueError("documents is required for Nvidia NIM rerank")

        query: Final = optional_rerank_params["query"]
        documents: Final = optional_rerank_params["documents"]

        # Transform query to object format
        query_obj: Final[NvidiaNimQueryObject] = {"text": query}

        # Transform documents to passages format
        passages: Final[list[NvidiaNimPassageObject]] = []
        for doc in documents:
            if isinstance(doc, str):
                passages.append({"text": doc})
            elif isinstance(doc, dict):
                # Preserve only the structured passage fields supported by the
                # selected rerank route.
                supported_fields: NvidiaNimPassageObject = {}  # mutable-ok: assembling a request TypedDict
                if "text" in self.SUPPORTED_PASSAGE_FIELDS and "text" in doc:
                    supported_fields["text"] = doc["text"]

View on GitHub (pinned to 6c2dcb801b)

Solutions

Pass documents=[...] alongside query.
Short-circuit upstream: if not documents, skip the rerank call entirely.
Check the exact spelling 'documents' in your kwargs.

Example fix

# before
results = litellm.rerank(model=m, query=q)  # forgot documents

# after
if not docs:
    return []
results = litellm.rerank(model=m, query=q, documents=docs)

Defensive patterns

Strategy: validation

Validate before calling

if not docs:
    return []  # nothing to rank — skip the provider call entirely

Type guard

def has_rankable_documents(value: object) -> bool:
    return isinstance(value, (list, tuple)) and len(value) > 0

Try / catch

try:
    litellm.rerank(model=m, query=q, documents=docs)
except ValueError as e:
    if "documents is required" in str(e):
        return []  # graceful no-op when retrieval found nothing
    raise

Prevention

Short-circuit on empty retrieval results before calling rerank.
Validate documents is a non-empty list at request intake.
Check the exact kwarg spelling 'documents' when porting code from Cohere-style APIs.

When it happens

Trigger: Calling litellm.rerank(model='nvidia_nim/...') with a query but no documents argument, or an empty/misspelled documents key (e.g. docs=[...]) so the parameter never reaches the transformer.

Common situations: Dynamic pipelines where the retrieval step returned nothing and documents was omitted instead of short-circuited, or parameter naming mismatched when porting from Cohere-style calls.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/a8fe84e723cdc077. Report an issue: GitHub.