What causes "query is required for HuggingFace rerank" in BerriAI/litellm?

BerriAI/litellm · error · ValueError

query is required for HuggingFace rerank

Error message

query is required for HuggingFace rerank

What it means

Raised by the HuggingFace rerank transformation when the translated rerank parameters lack a 'query' key. LiteLLM's huggingface rerank config expects the query/documents pair to have already been mapped to query/texts; a missing query aborts request building with a plain ValueError.

Source

Thrown at litellm/llms/huggingface/rerank/transformation.py:152

        }

        if api_key:
            default_headers["Authorization"] = f"Bearer {api_key}"

        if "Authorization" in headers:
            default_headers["Authorization"] = headers["Authorization"]

        return {**default_headers, **headers}

    def transform_rerank_request(
        self,
        model: str,
        optional_rerank_params: OptionalRerankParams | dict,
        headers: dict,
        litellm_params: dict | None = None,
    ) -> dict:
        if "query" not in optional_rerank_params:
            raise ValueError("query is required for HuggingFace rerank")
        if "texts" not in optional_rerank_params:
            raise ValueError("Cohere 'documents' param is required for HuggingFace rerank")
        # Ensure return_text is a boolean value
        # HuggingFace API expects return_text parameter, corresponding to our return_documents parameter
        request_body: Final = {
            "raw_scores": False,
            "truncate": False,
            "truncation_direction": "Right",
        }

        request_body.update(optional_rerank_params)

        return request_body

    def transform_rerank_response(
        self,
        model: str,
        raw_response: httpx.Response,

View on GitHub (pinned to 6c2dcb801b)

Solutions

Always pass query: litellm.rerank(model='huggingface/BAAI/bge-reranker-base', query='...', documents=[...]).
If writing a provider-agnostic layer, validate query is a non-empty string before dispatch.

Example fix

# before
litellm.rerank(model='huggingface/BAAI/bge-reranker-base', documents=docs)

# after
litellm.rerank(model='huggingface/BAAI/bge-reranker-base', query='what is python?', documents=docs)

Defensive patterns

Strategy: validation

Validate before calling

def validate_rerank_call(query: str | None, documents: list | None) -> None:
    if not query or not isinstance(query, str):
        raise ValueError('rerank requires a non-empty string query')
    if not documents:
        raise ValueError('rerank requires a non-empty documents list')

Type guard

def has_rerank_query(params: dict) -> bool:
    q = params.get('query')
    return isinstance(q, str) and len(q.strip()) > 0

Try / catch

try:
    litellm.rerank(model=model, query=q, documents=docs)
except ValueError as e:
    if 'query is required' in str(e):
        raise ValueError('caller bug: query missing in rerank dispatch') from e
    raise

Prevention

Validate query+documents at your API boundary, not per provider
Type rerank wrappers with (query: str, documents: list[str]) signatures

When it happens

Trigger: Calling the HF rerank transformation directly (e.g. custom integration calling transform_rerank_request) without a query; or calling litellm.rerank(model='huggingface/...') with query=None/omitted.

Common situations: Building a generic rerank wrapper over multiple providers and forgetting that huggingface requires an explicit query; passing only documents.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/37e877cd6fe61e69. Report an issue: GitHub.