BerriAI/litellm · error · ValueError

Nvidia NIM API key is required. Please set 'NVIDIA_NIM_API_K

Error message

Nvidia NIM API key is required. Please set 'NVIDIA_NIM_API_KEY' in your environment

What it means

Raised by litellm's Nvidia NIM rerank transformer in validate_environment when no API key can be resolved: api_key argument is None, NVIDIA_NIM_API_KEY env var is unset, and litellm.api_key is unset. Thrown before the rerank request is sent.

Source

Thrown at litellm/llms/nvidia_nim/rerank/transformation.py:163

        if non_default_params:
            optional_nvidia_nim_rerank_params.update(non_default_params)
        return dict(optional_nvidia_nim_rerank_params)

    def validate_environment(
        self,
        headers: dict,
        model: str,
        api_key: str | None = None,
        optional_params: dict | None = None,
    ) -> dict:
        """
        Validate that the Nvidia NIM API key is present.
        """
        if api_key is None:
            api_key = get_secret_str("NVIDIA_NIM_API_KEY") or litellm.api_key

        if api_key is None:
            raise ValueError("Nvidia NIM API key is required. Please set 'NVIDIA_NIM_API_KEY' in your environment")

        default_headers: Final = {
            "Authorization": f"Bearer {api_key}",
            "accept": "application/json",
            "content-type": "application/json",
        }

        # If 'Authorization' is provided in headers, it overrides the default
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: dict,

View on GitHub (pinned to 6c2dcb801b)

Solutions

export NVIDIA_NIM_API_KEY=<key from build.nvidia.com> (nvapi-...).
Or pass api_key explicitly to litellm.rerank().
For local/self-hosted NIM, pass a dummy api_key if the container enforces the header.
In litellm proxy, add api_key to the nvidia_nim model's litellm_params.

Example fix

# before
litellm.rerank(model="nvidia_nim/nv-rerankqa-mistral-4b-v3", query=q, documents=docs)

# after
litellm.rerank(
    model="nvidia_nim/nv-rerankqa-mistral-4b-v3",
    query=q,
    documents=docs,
    api_key=os.environ["NVIDIA_NIM_API_KEY"],
)

Defensive patterns

Strategy: validation

Validate before calling

import os
if not (os.getenv("NVIDIA_NIM_API_KEY") or litellm.api_key):
    raise RuntimeError("NVIDIA_NIM_API_KEY required for hosted nvidia_nim rerank")

Try / catch

try:
    litellm.rerank(model="nvidia_nim/nv-rerankqa-mistral-4b-v3", query=q, documents=docs)
except ValueError as e:
    if "NVIDIA_NIM_API_KEY" in str(e):
        raise RuntimeError("Config error: NVIDIA_NIM_API_KEY missing") from e
    raise

Prevention

Pass a dummy api_key for local NIM containers that require the header.
Keep hosted vs local NIM config in separate named profiles.
Preflight-check keys for every rerank provider you use.

When it happens

Trigger: Calling litellm.rerank() with model='nvidia_nim/...' without api_key, without NVIDIA_NIM_API_KEY in the environment, and without a global litellm.api_key — typical when targeting build.nvidia.com instead of a local NIM container.

Common situations: Switching from a local NIM deployment (no key needed) to NVIDIA's hosted endpoint (key required), missing secret in containers, or unset env var in notebooks.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/20a6b0334ae22aaf. Report an issue: GitHub.