BerriAI/litellm · critical · ValueError

FIREWORKS_API_KEY is not set. Please set 'FIREWORKS_API_KEY'

Error message

FIREWORKS_API_KEY is not set. Please set 'FIREWORKS_API_KEY' or 'FIREWORKS_AI_API_KEY' in your environment

What it means

The Fireworks rerank transformation validates the environment before proxying a rerank request. It looks for an explicit api_key, then FIREWORKS_API_KEY, then FIREWORKS_AI_API_KEY; when none resolves it raises this ValueError naming both accepted env vars. This is the rerank-specific analogue of the chat key check.

Source

Thrown at litellm/llms/fireworks_ai/rerank/transformation.py:108

            # Silently ignore max_chunks_per_doc as Fireworks AI doesn't support it
            pass

        if max_tokens_per_doc is not None:
            # Silently ignore max_tokens_per_doc as Fireworks AI doesn't support it
            pass

        return params

    def validate_environment(
        self,
        headers: dict,
        model: str,
        api_key: str | None = None,
        optional_params: dict | None = None,
    ) -> dict:
        api_key = self._get_api_key(api_key)
        if api_key is None:
            raise ValueError(
                "FIREWORKS_API_KEY is not set. Please set 'FIREWORKS_API_KEY' or 'FIREWORKS_AI_API_KEY' in your environment"
            )

        default_headers: Final = {
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        }

        # If 'Authorization' is provided in headers, it overrides the default.
        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        # Merge other headers, overriding any default ones except Authorization
        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,

View on GitHub (pinned to 6c2dcb801b)

Solutions

Export FIREWORKS_API_KEY (or FIREWORKS_AI_API_KEY) in the process that executes rerank.
Pass api_key explicitly on the rerank call or configure it on the litellm Router/model entry used for rerank.
If using litellm proxy, add the key to the rerank model's litellm_params in config.yaml.

Example fix

# before
results = litellm.rerank(model="fireworks_ai/fireworks/qwen3-reranker-8b", query="what is litellm", documents=["litellm is a sdk"])

# after
results = litellm.rerank(
    model="fireworks_ai/fireworks/qwen3-reranker-8b",
    query="what is litellm",
    documents=["litellm is a sdk"],
    api_key=os.environ["FIREWORKS_API_KEY"],
)

Defensive patterns

Strategy: validation

Validate before calling

import os

def validate_fireworks_rerank_env() -> None:
    if not (os.getenv("FIREWORKS_API_KEY") or os.getenv("FIREWORKS_AI_API_KEY")):
        raise RuntimeError("Rerank needs FIREWORKS_API_KEY or FIREWORKS_AI_API_KEY")

Try / catch

try:
    litellm.rerank(model="fireworks_ai/fireworks/qwen3-reranker-8b", query=q, documents=docs)
except ValueError as e:
    if "FIREWORKS_API_KEY is not set" in str(e):
        raise RuntimeError("Configure rerank credentials") from e
    raise

Prevention

In the litellm proxy, put the key in the rerank model's litellm_params so env leakage can't break it.
Use the same credential bootstrap for chat and rerank paths — one function, both checked.
Add a rerank integration test to CI, not just completion tests.

When it happens

Trigger: Calling litellm.rerank(model='fireworks_ai/...', query=..., documents=[...]) without api_key and without FIREWORKS_API_KEY / FIREWORKS_AI_API_KEY exported.

Common situations: Rerank added to an existing app after chat was configured, but only the chat path was tested; router/proxy config that supplies keys for completion models but not rerank models; local run where the env var lives only in the server's shell.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/e0a214702674600e. Report an issue: GitHub.