BerriAI/litellm · error · BaseLLMException

{error_message}

Error message

{error_message}

What it means

The rerank error-mapping hook: get_error_class wraps a provider's rerank HTTP error into BaseLLMException(status_code, message=error_message, headers). The literal '{error_message}' is the template — at runtime the exception carries the provider's actual error text, status code and headers from the failed rerank request.

Source

Thrown at litellm/llms/base_llm/rerank/transformation.py:93

    def map_cohere_rerank_params(
        self,
        non_default_params: dict,
        model: str,
        drop_params: bool,
        query: str,
        documents: list[str | dict[str, Any]],
        custom_llm_provider: str | None = None,
        top_n: int | None = None,
        rank_fields: list[str] | None = None,
        return_documents: bool | None = True,
        max_chunks_per_doc: int | None = None,
        max_tokens_per_doc: int | None = None,
        instruction: str | None = None,
    ) -> dict:
        pass

    def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:
        raise BaseLLMException(
            status_code=status_code,
            message=error_message,
            headers=headers,
        )

    def calculate_rerank_cost(
        self,
        model: str,
        custom_llm_provider: str | None = None,
        billed_units: RerankBilledUnits | None = None,
        model_info: ModelInfo | None = None,
    ) -> tuple[float, float]:
        """
        Calculates the cost per query for a given rerank model.

        Input:
            - model: str, the model name without provider prefix
            - custom_llm_provider: str, the provider used for the model. If provided, used to check if the litellm model info is for that provider.

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Catch BaseLLMException and branch on status_code: fix auth for 401/403, model name for 404, backoff for 429/5xx.
2. Verify the rerank provider env keys (e.g. COHERE_API_KEY) are set for the model in use.
3. Validate documents are non-empty strings/valid dicts before the call.
4. Check litellm proxy team settings allow the rerank route and model.

Example fix

# before
try:
    litellm.rerank(model='cohere/rerank-v3.5', query='q', documents=docs)
except Exception as e:
    print(e)

# after
from litellm.llms.base_llm.chat.transformation import BaseLLMException
try:
    litellm.rerank(model='cohere/rerank-v3.5', query='q', documents=docs)
except BaseLLMException as e:
    if e.status_code == 429:
        time.sleep(float(e.headers.get('retry-after', 1)) if e.headers else 1)
    else:
        raise

def validate_rerank_inputs(query: str, documents: list) -> None:
    if not query or not isinstance(query, str):
        raise ValueError('query must be a non-empty string')
    if not documents or not all(isinstance(d, (str, dict)) for d in documents):
        raise ValueError('documents must be non-empty strings or dicts')

def is_retryable_rerank_error(exc: BaseException) -> bool:
    return isinstance(exc, BaseLLMException) and exc.status_code in (429, 500, 502, 503, 504)

try:
    return litellm.rerank(model=model, query=query, documents=documents)
except BaseLLMException as e:
    if e.status_code == 429:
        time.sleep(2)
        return litellm.rerank(model=model, query=query, documents=documents)
    if e.status_code in (400, 404):
        raise ValueError(f'bad rerank request: {e.message}') from e
    raise

When it happens

Trigger: litellm.rerank() against a provider returning 4xx/5xx: bad/missing COHERE_API_KEY (401), wrong model name (404), rate limit (429), malformed documents array (400); the response handler converts the non-2xx into BaseLLMException.

Common situations: Missing provider API keys for rerank models; passing documents with mixed invalid types; free-tier rate limits; proxy rerank routes hitting an upstream outage.

Related errors

• {error_message}
• {error_message}
• Failed to fetch file '{file_path}': {e}
• Failed to list files in '{directory_path}': {e}
• Mavvrik FOCUS destination: failed to update metricsMarker ({

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/ed3c2aac6c1eb437. Report an issue: GitHub.