BerriAI/litellm · error · Exception

Failed to parse prompt response for '{prompt_id}': {e}

Error message

Failed to parse prompt response for '{prompt_id}': {e}

What it means

Raised by GenericPromptManager._fetch_prompt_from_api (sync path) when response.json() fails with json.JSONDecodeError — the prompt management API returned a 2xx response whose body is not valid JSON. This means the endpoint answered but is not serving the expected API (e.g. an HTML login page or a plain-text error page was returned).

Source

Thrown at litellm/integrations/generic_prompt_management/generic_prompt_manager.py:130

        params: Final = {
            "prompt_id": prompt_id,
            **(self.additional_provider_specific_query_params or {}),
        }
        http_client: Final = _get_httpx_client()

        try:
            response: Final = http_client.get(
                url,
                params=params,
                headers=self._get_headers(),
            )

            response.raise_for_status()
            return response.json()
        except httpx.HTTPError as e:
            raise Exception(f"Failed to fetch prompt '{prompt_id}' from API: {e}")
        except json.JSONDecodeError as e:
            raise Exception(f"Failed to parse prompt response for '{prompt_id}': {e}")

    async def async_fetch_prompt_from_api(
        self, prompt_id: str | None, prompt_spec: PromptSpec | None
    ) -> dict[str, Any]:
        """
        Fetch a prompt from the API asynchronously.
        """
        if prompt_id is None and prompt_spec is None:
            raise ValueError("prompt_id or prompt_spec is required")

        url: Final = f"{self.api_base}/beta/litellm_prompt_management"
        params: Final = {
            "prompt_id": prompt_id,
            **(
                prompt_spec.litellm_params.provider_specific_query_params
                if prompt_spec and prompt_spec.litellm_params.provider_specific_query_params
                else {}
            ),

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Confirm {api_base}/beta/litellm_prompt_management returns JSON: curl it directly and inspect Content-Type and body.
2. If an auth proxy is in front, make sure the api_key is passed in a way the proxy accepts (usually an Authorization header) so it forwards to the real API.
3. Adjust api_base to include the correct path prefix if the API is mounted under one (e.g. https://host/litellm).

import httpx

def assert_prompt_api_returns_json(api_base: str, prompt_id: str) -> None:
    resp = httpx.get(f"{api_base}/beta/litellm_prompt_management", params={"prompt_id": prompt_id}, timeout=10)
    ctype = resp.headers.get("content-type", "")
    if "json" not in ctype:
        raise RuntimeError(
            f"api_base is not serving JSON (content-type={ctype!r}); got an HTML/plain-text page instead"
        )

When it happens

Trigger: api_base points at a web server that returns HTML (a static site, a login redirect page, a reverse-proxy error page) with status 200; a CDN or WAF intercepting the request and returning an interstitial; the URL missing a path segment so the server serves its root page.

Common situations: Pointing api_base at a company portal domain instead of the actual API host; an nginx ingress configured to serve the frontend on the same path; an auth proxy that returns a 200 HTML login form when the session cookie is missing.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Failed to fetch prompt '{prompt_id}' from API: {e}
• Failed to fetch file '{file_path}' via JSON endpoint: {e}
• Error getting prompt from Humanloop: {e.response.text}
• Failed to transform Braintrust response: {str(e)}
• Error apply_db_fixes: {str(e)}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/447e96de3011759a. Report an issue: GitHub.