BerriAI/litellm · error · Exception

Failed to fetch prompt '{prompt_id}' from API: {e}

Error message

Failed to fetch prompt '{prompt_id}' from API: {e}

What it means

Raised by GenericPromptManager._fetch_prompt_from_api (sync path) when the HTTP GET to {api_base}/beta/litellm_prompt_management raises any httpx.HTTPError. This covers connection failures, timeouts, DNS errors, and non-2xx responses surfaced by response.raise_for_status(). The original exception is flattened into a message string, so the httpx exception type is lost.

Source

Thrown at litellm/integrations/generic_prompt_management/generic_prompt_manager.py:128

        url: Final = f"{self.api_base}/beta/litellm_prompt_management"
        params: Final = {
            "prompt_id": prompt_id,
            **(self.additional_provider_specific_query_params or {}),
        }
        http_client: Final = _get_httpx_client()

        try:
            response: Final = http_client.get(
                url,
                params=params,
                headers=self._get_headers(),
            )

            response.raise_for_status()
            return response.json()
        except httpx.HTTPError as e:
            raise Exception(f"Failed to fetch prompt '{prompt_id}' from API: {e}")
        except json.JSONDecodeError as e:
            raise Exception(f"Failed to parse prompt response for '{prompt_id}': {e}")

    async def async_fetch_prompt_from_api(
        self, prompt_id: str | None, prompt_spec: PromptSpec | None
    ) -> dict[str, Any]:
        """
        Fetch a prompt from the API asynchronously.
        """
        if prompt_id is None and prompt_spec is None:
            raise ValueError("prompt_id or prompt_spec is required")

        url: Final = f"{self.api_base}/beta/litellm_prompt_management"
        params: Final = {
            "prompt_id": prompt_id,
            **(
                prompt_spec.litellm_params.provider_specific_query_params
                if prompt_spec and prompt_spec.litellm_params.provider_specific_query_params

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Verify the API is reachable from where litellm runs: curl {api_base}/beta/litellm_prompt_management?prompt_id=<id> with the same api_key headers.
2. Fix the api_base value (scheme, host, port, path) in the model's litellm_params; it must be the root of a LiteLLM-compatible API.
3. If the status is 401/403, check that api_key is set and valid on the same litellm_params.
4. For transient failures, retry the request or wrap the completion call in a retry with backoff — the error is a plain Exception, so match on the message prefix 'Failed to fetch prompt'.

import httpx

def check_prompt_api_reachable(api_base: str, api_key: str | None, prompt_id: str) -> None:
    resp = httpx.get(
        f"{api_base}/beta/litellm_prompt_management",
        params={"prompt_id": prompt_id},
        headers={"Authorization": f"Bearer {api_key}"} if api_key else {},
        timeout=10,
    )
    resp.raise_for_status()
    assert resp.json(), "empty prompt response"

try:
    resp = litellm.completion(model="prompt-model", messages=msgs)
except Exception as e:
    if "Failed to fetch prompt" in str(e):
        # network/HTTP failure against the prompt API: retry with backoff
        handle_prompt_api_outage(e)
    else:
        raise

When it happens

Trigger: api_base points at a host that is down or unresolvable; a proxy/firewall blocks the egress request; the API returns 401/404/500 so raise_for_status() raises HTTPStatusError; the request exceeds the httpx client's default timeout; TLS certificate verification fails against a self-hosted API.

Common situations: Self-hosting the prompt API behind an internal network not reachable from the deployment environment; wrong api_base scheme (http vs https) or a missing port; expired API key causing 401s; transient network blips in Kubernetes pods during startup before the service endpoint is ready.

Related errors

• Failed to fetch file '{file_path}' via JSON endpoint: {e}
• Error from qdrant checking if /collections exist {collection
• Mavvrik FOCUS destination: register failed ({resp.status_cod
• Mavvrik FOCUS destination: failed to get signed URL ({resp.s
• Failed to parse prompt response for '{prompt_id}': {e}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/d72e3d0991decceb. Report an issue: GitHub.