Hmbown/CodeWhale · error · anyhow::Error

SSE stream idle timeout after

Error message

SSE stream idle timeout after {idle_secs}s — no data received (bytes_received={bytes}, stream_age_ms={age}, ms_since_last_event={since})

What it means

OpenAI-compatible streaming client idle-timeout: no bytes arrived on the SSE stream within the configured idle duration, so tokio_timeout returned Elapsed and the stream yields a diagnostic message including bytes received, stream age, and time since the last event, then aborts. This prevents a silently hung connection from stalling a turn indefinitely.

Solutions

1. Increase stream_idle_timeout for providers/models known to pause between chunks.
2. Restart or scale the backend provider; check its logs for the stalled generation.
3. Check proxy/ingress (nginx, Envoy) read timeouts and disable response buffering for SSE.
4. Retry the request — the failure is surfaced to the caller's existing retry path.

Example fix

// before
client = ChatClient::new().stream_idle_timeout(Duration::from_secs(60));
// after: tolerate slow local backends
client = ChatClient::new().stream_idle_timeout(Duration::from_secs(600));

// sanity-check idle timeout for slow local backends
assert!(idle >= Duration::from_secs(60), "idle timeout too aggressive");

match item {
    Err(e) if e.to_string().contains("idle timeout") => {
        warn!("provider stalled: {e}");
        retry_with_backoff();
    }
    other => handle(other)?,
}

When it happens

Trigger: handle_chat_completion_stream's byte_stream.next() times out: provider stops sending chunks mid-generation (overloaded server, dead upstream), a proxy holds the connection open without forwarding, or the model hangs before producing any output.

Common situations: Self-hosted/OpenAI-compatible backends (vLLM, Ollama, gateways) freezing under load; corporate proxies with aggressive SSE buffering; long tool-call or reasoning pauses exceeding the idle timeout.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• SSE stream idle timeout after
• SSE stream idle timeout after
• SSE stream idle timeout after
• SSE stream idle timeout after
• SSE stream idle timeout after

AI-assisted analysis of Hmbown/CodeWhale@433685b202 (2026-09-15). Data as JSON: /api/errors/b53b84b9f5280de3. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/src/client/chat.rs:1478

            // A number of OpenAI-compatible providers omit `[DONE]` but send a
            // terminal `finish_reason`. Either is valid terminal proof. A raw
            // HTTP EOF with neither is not: treating that as MessageStop turns
            // a truncated provider response into a successful empty turn.
            let mut saw_finish_reason = false;
            // Once an error has been emitted, do not follow it with a synthetic
            // MessageStop (or a second, less-specific premature-EOF error).
            let mut stream_failed = false;
            // Set when a complete line or unterminated flush failed UTF-8.
            // Skip further data-frame parsing so U+FFFD cannot enter the transcript.
            let mut decode_failed = false;

            'stream: loop {
                let chunk_result = match tokio_timeout(idle, byte_stream.next()).await {
                    Ok(Some(result)) => result,
                    Ok(None) => break, // Stream ended normally
                    Err(_elapsed) => {
                        stream_failed = true;
                        yield Err(anyhow::anyhow!(stream_idle_timeout_message(
                            idle,
                            bytes_received,
                            stream_start.elapsed(),
                            last_event_at.elapsed(),
                        )));
                        break;
                    }
                };
                let chunk = match chunk_result {
                    Ok(bytes) => bytes,
                    Err(e) => {
                        stream_failed = true;
                        // Walk the error source chain so reqwest's underlying
                        // hyper / h2 / io error is visible — without this the
                        // outer "error decoding response body" message tells
                        // us nothing about WHY the stream died.
                        let mut error_chain = format!("{e}");
                        let mut current: Option<&(dyn std::error::Error + 'static)> =

View on GitHub (pinned to 433685b202)