BerriAI/litellm · error · HuggingFaceError

{embeddings[error]}

Error message

{embeddings[error]}

What it means

Raised in the sync embedding path after the HTTP call succeeds at the transport level but the HuggingFace response body is a JSON object containing an 'error' key. Litellm surfaces the upstream error text verbatim with status 500 (regardless of the upstream code).

Source

Thrown at litellm/llms/huggingface/embedding/handler.py:278

        if client is None:
            client = get_async_httpx_client(
                llm_provider=litellm.LlmProviders.HUGGINGFACE,
            )

        response: Final = await client.post(api_base, headers=headers, data=json.dumps(data))

        ## LOGGING
        logging_obj.post_call(
            input=input,
            api_key=api_key,
            additional_args={"complete_input_dict": data},
            original_response=response,
        )

        embeddings: Final = response.json()

        if "error" in embeddings:
            raise HuggingFaceError(status_code=500, message=embeddings["error"])

        ## PROCESS RESPONSE ##
        return self._process_embedding_response(
            embeddings=embeddings,
            model_response=model_response,
            model=model,
            input=input,
            encoding=encoding,
        )

    def embedding(
        self,
        model: str,
        input: list,
        model_response: EmbeddingResponse,
        optional_params: dict,
        litellm_params: dict,
        logging_obj: LiteLLMLoggingObj,

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read the surfaced message — it is HF's own error text and names the real cause (auth, loading, limits).
2. Verify the token: set HF_TOKEN / pass api_key with access to the model (check gated-model acceptance).
3. Confirm the model id exists and serves the embedding task on the Hub; try a known-good model to isolate.
4. Retry after a short backoff for transient 'currently loading' responses from serverless inference.

def huggingface_ready(model: str, api_key: str) -> bool:
    import httpx
    r = httpx.get(f'https://huggingface.co/api/models/{model}', headers={'Authorization': f'Bearer {api_key}'})
    return r.status_code == 200

from tenacity import retry, wait_exponential, retry_if_exception

@retry(wait=wait_exponential(multiplier=1, max=30), retries=3,
       retry=retry_if_exception(lambda e: 'loading' in str(e).lower()))
def embed(texts):
    return litellm.embedding(model=MODEL, input=texts)

When it happens

Trigger: HF Inference API returns {"error": ...} for the embedding request — e.g. model is gated/private, invalid API token, model loading or unavailable on the inference endpoint, or input too long for the model context.

Common situations: Expired/missing HF_TOKEN, using a model id that was renamed or deprecated on the Hub, cold-start/loading errors on serverless inference, or a free-tier rate limit returning an error payload.

Related errors

• message (upstream DashScope response error message)
• {completion_response[error]}
• api_key is None. Please set AZURE_AI_API_KEY or dynamically
• DashScope API key is required. Set 'DASHSCOPE_API_KEY' env v
• response_json.get("message", str(response_json)) (upstream D

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/407d2b57a3ff0c7e. Report an issue: GitHub.