BerriAI/litellm · error · VoyageError

Failed to parse response: {raw_response.text}

Error message

Failed to parse response: {raw_response.text}

What it means

In VoyageRerankConfig.transform_rerank_response, non-200 statuses raise VoyageError first; for a 200 whose body still fails raw_response.json(), this VoyageError('Failed to parse response: ...') is raised with the raw text embedded. Typical causes: an HTML/empty body from a proxy or gateway, a truncated stream, or a non-JSON success response.

Source

Thrown at litellm/llms/voyage/rerank/transformation.py:98

        self,
        model: str,
        raw_response: httpx.Response,
        model_response: RerankResponse,
        logging_obj: LiteLLMLoggingObj,
        api_key: str | None = None,
        request_data: dict = {},
        optional_params: dict = {},
        litellm_params: dict = {},
    ) -> RerankResponse:
        if raw_response.status_code != 200:
            raise VoyageError(message=raw_response.text, status_code=raw_response.status_code)

        logging_obj.post_call(original_response=raw_response.text)

        try:
            _json_response: Final = raw_response.json()
        except Exception:
            raise VoyageError(
                message=f"Failed to parse response: {raw_response.text}",
                status_code=raw_response.status_code,
            )

        # Voyage AI returns results in "data" key, not "results"
        _results: Final[list[dict] | None] = _json_response.get("data")
        if _results is None:
            raise ValueError(f"No results found in the response={_json_response}")

        # Transform to LiteLLM format
        transformed_results: Final = []
        for result in _results:
            transformed_result: dict[str, Any] = {
                "index": result["index"],
                "relevance_score": result["relevance_score"],
            }
            if "document" in result:
                if isinstance(result["document"], str):

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Read the embedded raw text in the exception message to see what the server actually returned.
2. curl -sS https://api.voyageai.com/v1/rerank -H 'Authorization: Bearer $VOYAGE_API_KEY' ... to confirm the endpoint returns JSON directly.
3. Remove or bypass HTTP(S)_PROXY for the Voyage host; add proxy exceptions.
4. Retry on this specific error - truncated/interstitial bodies are often transient.

Example fix

# before (no handling; proxy sometimes returns HTML with 200)
result = litellm.rerank(model="voyage/voyage-3-rerank", query=q, documents=docs)

# after (retry on parse failure)
from litellm.llms.voyage.common_utils import VoyageError

for attempt in range(3):
    try:
        result = litellm.rerank(model="voyage/voyage-3-rerank", query=q, documents=docs)
        break
    except VoyageError as e:
        if attempt == 2 or not str(e).startswith("Failed to parse response"):
            raise

from tenacity import retry, retry_if_exception, stop_after_attempt, wait_exponential
from litellm.llms.voyage.common_utils import VoyageError

def _is_parse_error(e: Exception) -> bool:
    return isinstance(e, VoyageError) and str(e).startswith("Failed to parse response")

@retry(retry=retry_if_exception(_is_parse_error), stop=stop_after_attempt(3), wait=wait_exponential(1, 2, 10))
def voyage_rerank(query: str, documents: list[str]):
    return litellm.rerank(model="voyage/voyage-3-rerank", query=query, documents=documents)

When it happens

Trigger: litellm.rerank(model="voyage/voyage-3-rerank", query=..., documents=[...]) routed through a corporate proxy that rewrites responses; intermittent truncated bodies on long document lists; a misconfigured api_base serving a landing page with 200 status.

Common situations: Self-hosted gateways in front of api.voyageai.com; retry storms producing gateway interstitials; debugging without verbose logs so the embedded raw text is missed; rate-limiter middlewares returning empty 200s.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• No results found in the response={_json_response}
• No results found in the response={response}
• Missing required fields in the result={result}
• No results found in the response={response}
• Missing required fields in the result={result}

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/3547f8344897556c. Report an issue: GitHub.