BerriAI/litellm · error · TextCompletionCodestralError

{e}

Error message

{e}

What it means

The catch-all branch in the async Codestral handler: any exception from async_handler.post that is not an httpx.HTTPStatusError (connect errors, timeouts, DNS failures, TLS errors) is re-raised as TextCompletionCodestralError with status_code=500 and the original exception string as the message. The 500 is local — no server response exists.

Source

Thrown at litellm/llms/codestral/completion/handler.py:359

        optional_params: dict,
        timeout: float | httpx.Timeout,
        litellm_params=None,
        logger_fn=None,
        headers={},
    ) -> TextCompletionResponse:
        async_handler: Final = get_async_httpx_client(
            llm_provider=litellm.LlmProviders.TEXT_COMPLETION_CODESTRAL,
            params={"timeout": timeout},
        )
        try:
            response: Final = await async_handler.post(api_base, headers=headers, data=json.dumps(data))
        except httpx.HTTPStatusError as e:
            raise TextCompletionCodestralError(
                status_code=e.response.status_code,
                message=f"HTTPStatusError - {e.response.text}",
            )
        except Exception as e:
            raise TextCompletionCodestralError(
                status_code=500, message=f"{e}"
            )  # don't use verbose_logger.exception, if exception is raised
        return self.process_text_completion_response(
            model=model,
            response=response,
            model_response=model_response,
            stream=stream,
            logging_obj=logging_obj,
            api_key=api_key,
            data=data,
            messages=messages,
            print_verbose=print_verbose,
            optional_params=optional_params,
            encoding=encoding,
        )

    @track_llm_api_timing()
    async def async_streaming(

View on GitHub (pinned to 6c2dcb801b)

Solutions

  1. Read error.message: 'timed out' means raise the timeout param; 'connect'/'name resolution' means network/DNS/firewall.
  2. Increase timeout on the call (litellm text_completion supports timeout=...).
  3. Fix egress: allow api.mistral.ai:443 from the runtime; verify with curl from the same host.
  4. Retry transient network failures with exponential backoff.

Example fix

# before
resp = await litellm.atext_completion(
    model="text-completion-codestral/codestral-latest", prompt=prompt, timeout=10
)

# after
resp = await litellm.atext_completion(
    model="text-completion-codestral/codestral-latest", prompt=prompt, timeout=120
)
Defensive patterns

Strategy: retry

Validate before calling

import socket

# fail fast on unreachable endpoint before launching async work
socket.getaddrinfo("api.mistral.ai", 443)  # raises on DNS failure

Try / catch

from litellm.exceptions import TextCompletionCodestralError

try:
    resp = await litellm.atext_completion(..., timeout=120)
except TextCompletionCodestralError as e:
    if e.status_code == 500 and ("timed out" in str(e.message) or "connect" in str(e.message).lower()):
        await retry_with_jitter()  # network-level, no server involved
    else:
        raise

Prevention

When it happens

Trigger: Network-level failures during the async POST: httpx.ConnectError, httpx.ConnectTimeout, httpx.ReadTimeout, DNS resolution failure, TLS certificate errors, or a misconfigured api_base hostname.

Common situations: Short timeouts with long Codestral generations (ReadTimeout); egress firewall blocking api.mistral.ai; DNS problems in containers; self-signed proxy certificates. Distinguish from a real server 500 by the message text (it contains the httpx exception repr, not a JSON body).

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/22bb7f558da88512. Report an issue: GitHub.