BerriAI/litellm · error · ValueError

Nvidia NIM API key is required, please set 'NVIDIA_NIM_API_K

Error message

Nvidia NIM API key is required, please set 'NVIDIA_NIM_API_KEY' in your environment

What it means

Credential resolution guard for NVIDIA NIM rerank: no API key was resolvable from the call, optional params, or the NVIDIA_NIM_API_KEY environment variable, so the rerank request cannot be authenticated.

Source

Thrown at litellm/rerank_api/main.py:316

            response = base_llm_http_handler.rerank(
                model=model,
                custom_llm_provider=_custom_llm_provider,
                optional_rerank_params=optional_rerank_params,
                logging_obj=litellm_logging_obj,
                provider_config=rerank_provider_config,
                timeout=optional_params.timeout,
                api_key=dynamic_api_key or optional_params.api_key,
                api_base=api_base,
                _is_async=_is_async,
                headers=headers or litellm.headers or {},
                client=client,
                model_response=model_response,
                litellm_params=rerank_litellm_params,
            )
        elif _custom_llm_provider == litellm.LlmProviders.NVIDIA_NIM:
            if dynamic_api_key is None:
                raise ValueError("Nvidia NIM API key is required, please set 'NVIDIA_NIM_API_KEY' in your environment")

            # Note: For rerank, the base URL is different from chat/embeddings
            # Rerank uses ai.api.nvidia.com instead of integrate.api.nvidia.com
            api_base = (
                optional_params.api_base
                or get_secret("NVIDIA_NIM_API_BASE")
                or "https://ai.api.nvidia.com"  # Default for rerank
            )

            response = base_llm_http_handler.rerank(
                model=model,
                custom_llm_provider=_custom_llm_provider,
                optional_rerank_params=optional_rerank_params,
                logging_obj=litellm_logging_obj,
                provider_config=rerank_provider_config,
                timeout=optional_params.timeout,
                api_key=dynamic_api_key or optional_params.api_key,
                api_base=api_base,

View on GitHub (pinned to 77b7c6c40c)

Solutions

  1. Set NVIDIA_NIM_API_KEY in your environment.
Defensive patterns

Strategy: validation

When it happens

Trigger: Thrown at litellm/rerank_api/main.py:316 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/144999acf4cd4158. Report an issue: GitHub.