run-llama/llama_index · error · ValueError

Must pass in cohere api key or specify via COHERE_API_KEY en

Error message

Must pass in cohere api key or specify via COHERE_API_KEY environment variable

What it means

Raised by CohereRerankRelevancyMetric.__init__ when no Cohere API key is supplied: the constructor falls back to the COHERE_API_KEY environment variable, and when neither is present it tries to raise this ValueError. Note a latent bug: the code catches IndexError, but os.environ[...] raises KeyError when the variable is missing, so in practice you will usually see KeyError('COHERE_API_KEY') instead of this ValueError.

Source

Thrown at llama-index-core/llama_index/core/evaluation/retrieval/metrics.py:446


class CohereRerankRelevancyMetric(BaseRetrievalMetric):
    """Cohere rerank relevancy metric."""

    metric_name: ClassVar[str] = "cohere_rerank_relevancy"
    model: str = Field(description="Cohere model name.")

    _client: Any = PrivateAttr()

    def __init__(
        self,
        model: str = "rerank-english-v2.0",
        api_key: Optional[str] = None,
    ):
        try:
            api_key = api_key or os.environ["COHERE_API_KEY"]
        except IndexError:
            raise ValueError(
                "Must pass in cohere api key or "
                "specify via COHERE_API_KEY environment variable "
            )
        try:
            from cohere import Client  # pants: no-infer-dep
        except ImportError:
            raise ImportError(
                "Cannot import cohere package, please `pip install cohere`."
            )

        super().__init__(model=model)
        self._client = Client(api_key=api_key)

    def _get_agg_func(self, agg: Literal["max", "median", "mean"]) -> Callable:
        """Get agg func."""
        return _AGG_FUNC[agg]

    def compute(

View on GitHub (pinned to afd0fef371)

Solutions

Pass the key explicitly: CohereRerankRelevancyMetric(api_key=os.environ['COHERE_API_KEY']) after exporting it in the shell (export COHERE_API_KEY=...).
Set the variable in the environment that launches the process (docker-compose env, CI secret, systemd Environment=) rather than relying on the constructor.
If you see KeyError instead of this ValueError, that is the upstream catch-the-wrong-exception bug (IndexError vs KeyError) — catch KeyError too or supply api_key explicitly to bypass it.

Example fix

// before
metric = CohereRerankRelevancyMetric()  # COHERE_API_KEY unset

# after
export COHERE_API_KEY=...   # or:
metric = CohereRerankRelevancyMetric(api_key=get_cohere_key())

Defensive patterns

Strategy: validation

Validate before calling

import os

cohere_key = os.environ.get("COHERE_API_KEY")
if not cohere_key and not explicit_api_key:
    raise RuntimeError("Configure COHERE_API_KEY before building CohereRerankRelevancyMetric")

Try / catch

try:
    metric = CohereRerankRelevancyMetric(api_key=cohere_key)
except ValueError as e:
    if "cohere api key" in str(e).lower():
        # handle missing credentials
    else:
        raise
# NOTE: due to the IndexError/KeyError bug, also catch KeyError('COHERE_API_KEY')

Prevention

Export COHERE_API_KEY in the launching environment (compose env, CI secret, shell profile) rather than relying on the constructor fallback.
Assert required env vars at process startup with a fast-fail checklist.
Pass api_key explicitly from your own secret manager so library fallback paths are never exercised.

When it happens

Trigger: Instantiating CohereRerankRelevancyMetric(model=..., api_key=None) (or omitting api_key) in a process where the COHERE_API_KEY environment variable is not set — e.g. building a RetrieverEvaluator with metrics=['cohere_rerank_relevancy'] via resolve_metrics without configuring Cohere credentials.

Common situations: CI runs or containers where secrets are not exported; local shells where the key was set in a different terminal; .env files loaded after module import; the IndexError/KeyError mismatch surfacing after a Python or llama-index upgrade makes the intended ValueError unreachable.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/844fa4e3515d7e59. Report an issue: GitHub.