What causes "Failed to connect to Braintrust API: {str(e)}" in BerriAI/litellm?

Wrap the async for loop in try/except OpenAIError and inspect .status_code/.message.

BerriAI/litellm · error · HTTPException

Failed to connect to Braintrust API: {str(e)}

Error message

Failed to connect to Braintrust API: {str(e)}

What it means

The async counterpart of the sync streaming error: raised when iterating an async text-completion stream (async for transformed_chunk in streamwrapper) throws. Exceptions during async chunk delivery - connection resets, provider error events, parsing failures - are caught and re-raised as OpenAIError with upstream status_code, headers, and text. The 'error_text' placeholder message means the caught exception had no .text attribute, so str(e) was used.

Source

Thrown at cookbook/litellm_proxy_server/braintrust_prompt_wrapper_server.py:216

        "Authorization": f"Bearer {braintrust_token}",
        "Accept": "application/json",
    }
    print(f"headers: {headers}")
    print(f"braintrust_url: {braintrust_url}")
    print(f"braintrust_token: {braintrust_token}")

    try:
        async with httpx.AsyncClient(timeout=30.0) as client:
            response = await client.get(braintrust_url, headers=headers)
            response.raise_for_status()
            braintrust_data = response.json()
    except httpx.HTTPStatusError as e:
        raise HTTPException(
            status_code=e.response.status_code,
            detail=f"Braintrust API error: {e.response.text}",
        )
    except httpx.RequestError as e:
        raise HTTPException(
            status_code=502,
            detail=f"Failed to connect to Braintrust API: {str(e)}",
        )
    except json.JSONDecodeError as e:
        raise HTTPException(
            status_code=502,
            detail=f"Failed to parse Braintrust API response: {str(e)}",
        )

    print(f"braintrust_data: {braintrust_data}")
    # Transform the response
    try:
        transformed_data = transform_braintrust_response(braintrust_data)
        print(f"transformed_data: {transformed_data}")
        return JSONResponse(content=transformed_data)
    except Exception as e:
        raise HTTPException(
            status_code=500,

View on GitHub (pinned to 6c2dcb801b)

Solutions

Wrap the async for loop in try/except OpenAIError and inspect .status_code/.message.
Set timeout and num_retries on the async call; ensure the event loop is not blocked elsewhere, starving the stream reader.
For recurring mid-stream drops, increase client timeouts and check proxy idle-timeout settings.
Degrade gracefully to a non-streaming call on repeated stream failures if partial output is unacceptable.

Example fix

# before
async for chunk in await litellm.atext_completion(model=..., prompt=..., stream=True):
    await handle(chunk)

# after
try:
    async for chunk in await litellm.atext_completion(model=..., prompt=..., stream=True):
        await handle(chunk)
except litellm.exceptions.OpenAIError as e:
    logger.error("async stream failed: %s (%s)", e.message, e.status_code)

Defensive patterns

Strategy: try-catch

Type guard

from litellm.exceptions import OpenAIError

def is_async_stream_error(e: BaseException) -> bool:
    return isinstance(e, OpenAIError) and getattr(e, "status_code", None) is not None

Try / catch

from litellm.exceptions import OpenAIError

try:
    async for chunk in await litellm.atext_completion(model=..., prompt=..., stream=True):
        await handle(chunk)
except OpenAIError as e:
    logger.error("async stream failed: %s (%s)", e.message, e.status_code)
    raise

Prevention

Keep the event loop unblocked so the async stream reader is serviced promptly.
Set explicit async timeouts; do not rely on defaults for long generations.
Fall back to non-streaming async calls when streams repeatedly fail.

When it happens

Trigger: Using await litellm.atext_completion(..., stream=True) (or the async iterator) when the aiohttp/httpx connection drops mid-stream, the provider emits an error event after headers were sent, or a chunk violates the expected schema during async parsing.

Common situations: Async web servers (FastAPI) proxying streams under load; event-loop friendly code hitting the same mid-stream failures as sync but surfacing CancelledError-adjacent connection errors; timeouts from async clients with defaults too small.

Related errors

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/65e1a3d4bb6846df. Report an issue: GitHub.