BerriAI/litellm · error · InfinityError

{raw_response.text}

Error message

{raw_response.text}

What it means

Raised when the Infinity embedding server's response body is not valid JSON. The raw body text and HTTP status are wrapped in InfinityError — typical when the base URL points at a wrong route (HTML page), the service is down, or a proxy intercepts the request.

Source

Thrown at litellm/llms/infinity/embedding/transformation.py:120

            "model": model,
            **optional_params,
        }

    def transform_embedding_response(
        self,
        model: str,
        raw_response: httpx.Response,
        model_response: EmbeddingResponse,
        logging_obj: LiteLLMLoggingObj,
        api_key: str | None = None,
        request_data: dict = {},
        optional_params: dict = {},
        litellm_params: dict = {},
    ) -> EmbeddingResponse:
        try:
            raw_response_json: Final = raw_response.json()
        except Exception:
            raise InfinityError(message=raw_response.text, status_code=raw_response.status_code)

        # model_response.usage
        model_response.model = raw_response_json.get("model")
        model_response.data = raw_response_json.get("data")
        model_response.object = raw_response_json.get("object")

        usage: Final = Usage(
            prompt_tokens=raw_response_json.get("usage", {}).get("prompt_tokens", 0),
            total_tokens=raw_response_json.get("usage", {}).get("total_tokens", 0),
        )
        model_response.usage = usage
        return model_response

    def get_error_class(self, error_message: str, status_code: int, headers: dict | httpx.Headers) -> BaseLLMException:
        return InfinityError(message=error_message, status_code=status_code, headers=headers)

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Check the embedded body text — it usually names the real endpoint/problem.
2. Confirm Infinity is running: curl http://<api_base>/models.
3. Correct api_base host/port in the client or proxy config.

def infinity_alive(base: str) -> bool:
    import httpx
    try:
        return httpx.get(f'{base.rstrip("/")}/models', timeout=5).status_code == 200
    except Exception:
        return False

try:
    resp = litellm.embedding(model='infinity/m', input=texts, api_base=base)
except Exception as e:
    if 'InfinityError' in type(e).__name__ and ('<html' in str(e) or 'not connect' in str(e)):
        raise RuntimeError(f'Infinity at {base} unreachable or not JSON; check service') from e
    raise

When it happens

Trigger: api_base pointing at a non-Infinity route or wrong port (e.g. missing /embeddings suffix is fine since it's appended, but hitting the docs UI), the Infinity container not started, or auth middleware returning HTML.

Common situations: Local dev where Infinity runs on a different port than configured; service crashed mid-request; URL with a typo'd path prefix.

Related errors

• {raw_response.text}
• api_base is required for Infinity embeddings
• Failed to parse raw Azure embedding response: {json_error}
• Failed to parse DashScope response as JSON: {e}
• Original Response received: {raw_response.text}

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/d87b669da8a33d4d. Report an issue: GitHub.