{"record":{"id":"1ac212e47ca5868b","repo":"BerriAI/litellm","slug":"query-is-required-for-nvidia-nim-rerank","errorCode":null,"errorMessage":"query is required for Nvidia NIM rerank","messagePattern":"query is required for Nvidia NIM rerank","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/llms/nvidia_nim/rerank/transformation.py","lineNumber":198,"sourceCode":"        model: str,\n        optional_rerank_params: dict,\n        headers: dict,\n        litellm_params: dict | None = None,\n    ) -> dict:\n        \"\"\"\n        Transform request to Nvidia NIM format.\n\n        Nvidia NIM expects:\n        - query as {text: \"...\"}\n        - documents as passages: [{text: \"...\"}, ...]\n        - Optional: truncate (NONE or END), top_k\n\n        Note: optional_rerank_params may contain provider-specific params like 'top_k' and 'truncate'\n        that aren't in the OptionalRerankParams TypedDict but are passed through at runtime.\n        The mapping from Cohere's 'top_n' to Nvidia's 'top_k' already happened in map_cohere_rerank_params.\n        \"\"\"\n        if \"query\" not in optional_rerank_params:\n            raise ValueError(\"query is required for Nvidia NIM rerank\")\n        if \"documents\" not in optional_rerank_params:\n            raise ValueError(\"documents is required for Nvidia NIM rerank\")\n\n        query: Final = optional_rerank_params[\"query\"]\n        documents: Final = optional_rerank_params[\"documents\"]\n\n        # Transform query to object format\n        query_obj: Final[NvidiaNimQueryObject] = {\"text\": query}\n\n        # Transform documents to passages format\n        passages: Final[list[NvidiaNimPassageObject]] = []\n        for doc in documents:\n            if isinstance(doc, str):\n                passages.append({\"text\": doc})\n            elif isinstance(doc, dict):\n                # Preserve only the structured passage fields supported by the\n                # selected rerank route.\n                supported_fields: NvidiaNimPassageObject = {}  # mutable-ok: assembling a request TypedDict","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/BerriAI/litellm/blob/6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d/litellm/llms/nvidia_nim/rerank/transformation.py#L180-L216","documentation":"Raised by litellm's Nvidia NIM rerank transformer in transform_rerank_request when 'query' is absent from optional_rerank_params. The native NIM /v1/ranking endpoint requires a query; litellm maps query to {text: query} and refuses to build a request without it.","triggerScenarios":"Calling litellm.rerank(model='nvidia_nim/...') without the query argument, or passing it under a different name (e.g. q=, search_query=) so it never reaches optional_rerank_params.","commonSituations":"Adapting code from another rerank API with different parameter names, or building kwargs dynamically and accidentally dropping 'query'.","solutions":["Always pass query explicitly: litellm.rerank(model=..., query='...', documents=[...]).","Check your kwargs builder includes the literal key 'query'.","Validate required keys before calling (see defense code)."],"exampleFix":"# before\nlitellm.rerank(model=\"nvidia_nim/nv-rerankqa-mistral-4b-v3\", documents=docs, q=\"what is litellm?\")\n\n# after\nlitellm.rerank(model=\"nvidia_nim/nv-rerankqa-mistral-4b-v3\", documents=docs, query=\"what is litellm?\")","handlingStrategy":"type-guard","validationCode":"def validate_rerank_args(query: str | None, documents: list | None) -> None:\n    if not query or not query.strip():\n        raise ValueError(\"query is required for rerank\")\n    if not documents:\n        raise ValueError(\"documents is required for rerank\")","typeGuard":"from typing import Any, TypeGuard\n\ndef is_rerank_ready(documents: Any) -> TypeGuard[list[str]]:\n    return isinstance(documents, list) and len(documents) > 0 and all(isinstance(d, str) and d.strip() for d in documents)","tryCatchPattern":"try:\n    litellm.rerank(model=m, query=q, documents=docs)\nexcept ValueError as e:\n    if \"query is required\" in str(e):\n        raise HTTPException(400, \"missing query\") from e\n    raise","preventionTips":["Use the literal kwarg names query= and documents= for all rerank calls.","Reject empty queries/documents at your own API boundary.","Write one shared rerank wrapper in your codebase so params are always set correctly."],"tags":["nvidia-nim","rerank","missing-parameter","query"],"backgroundTag":null,"analyzedSha":"6c2dcb801bf2b75c18f1bb24140e7cf57465cc4d","analyzedAt":"2026-08-15T07:12:03.035Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}