What causes "Failed to fetch provider mapping: {e}" in BerriAI/litellm?

Check the embedded status code: 401/403 → fix HUGGINGFACE_API_KEY; 404 → wrong model id; 429/5xx → back off and retry later

BerriAI/litellm · error · HuggingFaceError

Failed to fetch provider mapping

Error message

Failed to fetch provider mapping: {e}

What it means

While fetching the provider mapping from the HF Hub API, any httpx.HTTPError (connection failure, DNS error, or a non-2xx status via raise_for_status) is converted into HuggingFaceError with message 'Failed to fetch provider mapping: <e>' and the upstream status code (500 when there is no response object). This is a transport/upstream failure talking to huggingface.co, not a verdict on your model.

Solutions

Check the embedded status code: 401/403 → fix HUGGINGFACE_API_KEY; 404 → wrong model id; 429/5xx → back off and retry later
Verify basic connectivity from the same host: curl 'https://huggingface.co/api/models/<model>?expand=inferenceProviderMapping'
Configure proxy env vars (HTTPS_PROXY) if egress goes through a proxy
If the environment cannot reach the Hub at all, bypass routing by passing api_base to a self-hosted endpoint
Retry transient failures — the mapping fetch is not retried internally

Example fix

# before
litellm.completion(model="hf/hf-inference/meta-llama/Llama-3.1-8B-Instruct", messages=msgs)  # no network to hf.co

# after — self-hosted endpoint, no Hub lookup needed
litellm.completion(
    model="hf/meta-llama/Llama-3.1-8B-Instruct",
    api_base="https://xxxx.endpoints.huggingface.cloud/v1",
    api_key=os.environ["HUGGINGFACE_API_KEY"],
    messages=msgs,
)

Defensive patterns

Strategy: try-catch

Try / catch

Catch HuggingFaceError around the completion/embedding call and branch on status_code: 429/5xx and connection errors → retry with exponential backoff and jitter (the Hub fetch is not internally retried); 401/403 → fix the token and stop; 404 → wrong model id, do not retry. Optionally fall back to a self-hosted api_base when the Hub is unreachable.

Prevention

Cache provider mappings in your own layer with a TTL instead of relying on every cold process hitting the Hub
Run from networks with verified egress to huggingface.co, or pre-configure api_base in locked-down environments
Set a valid HUGGINGFACE_API_KEY and monitor for 401s that indicate token expiry

When it happens

Trigger: Network egress blocked from the runtime; DNS failure; Hub returning 401/403 (bad token), 404 (repo does not exist), 429 (rate limited), or 5xx. The lookup happens lazily on the first 'hf/<provider>/<model>' call without api_base, and is lru_cached afterwards.

Common situations: Corporate proxies/firewalls blocking huggingface.co; air-gapped or offline environments; an invalid HUGGINGFACE_API_KEY causing 401 on every call; Hub outages; CI runners with restricted networking.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/6becea2bc4229e9c. Report an issue: GitHub.

Appendix: source

Thrown at litellm/llms/huggingface/common_utils.py:98

    params: Final = {"expand": ["inferenceProviderMapping"]}

    try:
        response: Final = httpx.get(path, headers=headers, params=params)
        response.raise_for_status()
        provider_mapping: Final = response.json().get("inferenceProviderMapping")

        if provider_mapping is None:
            raise ValueError(f"No provider mapping found for model {model}")

        return provider_mapping
    except httpx.HTTPError as e:
        if hasattr(e, "response"):
            status_code = getattr(e.response, "status_code", 500)
            headers = getattr(e.response, "headers", {})
        else:
            status_code = 500
            headers = {}
        raise HuggingFaceError(
            message=f"Failed to fetch provider mapping: {e}",
            status_code=status_code,
            headers=headers,
        )

View on GitHub (pinned to 77b7c6c40c)