aaif-goose/goose · error · anyhow::Error

Ollama stream stalled: no data received for {}s. This may in

Error message

Ollama stream stalled: no data received for {}s. This may indicate the model is overwhelmed by the request payload. Try a smaller model, reduce the number of tools, or increase the timeout via OLLAMA_STREAM_TIMEOUT, GOOSE_STREAM_TIMEOUT, or OLLAMA_TIMEOUT in your config.

What it means

goose wraps the Ollama SSE stream with a per-chunk watchdog: after the first SSE line arrives, every subsequent line must arrive within a timeout or the stream aborts with this error. The budget resolves from OLLAMA_STREAM_TIMEOUT, then GOOSE_STREAM_TIMEOUT, then OLLAMA_TIMEOUT, else the 120-second default (OLLAMA_DEFAULT_CHUNK_TIMEOUT_SECS). Time-to-first-token is deliberately exempt; only gaps between chunks count. It fires when the server stops sending mid-generation — typically an overwhelmed local model, OOM/crash of the ollama process, or a network/proxy drop.

Source

Thrown at crates/goose-providers/src/ollama.rs:495

    stream: impl futures::Stream<Item = anyhow::Result<String>> + Unpin + Send + 'static,
    timeout_secs: u64,
) -> std::pin::Pin<Box<dyn futures::Stream<Item = anyhow::Result<String>> + Send>> {
    let timeout = Duration::from_secs(timeout_secs);
    Box::pin(try_stream! {
        let mut stream = stream;

        // Allow time-to-first-token to be governed by the request timeout.
        // Only enforce per-chunk timeout after first SSE line arrives.
        match stream.next().await {
            Some(first_item) => yield first_item?,
            None => return,
        }
        loop {
            match tokio::time::timeout(timeout, stream.next()).await {
                Ok(Some(item)) => yield item?,
                Ok(None) => break,
                Err(_) => {
                    Err::<(), anyhow::Error>(anyhow::anyhow!(
                        "Ollama stream stalled: no data received for {}s. \
                         This may indicate the model is overwhelmed by the request payload. \
                         Try a smaller model, reduce the number of tools, or increase the \
                         timeout via OLLAMA_STREAM_TIMEOUT, GOOSE_STREAM_TIMEOUT, or \
                         OLLAMA_TIMEOUT in your config.",
                        timeout_secs
                    ))?;
                }
            }
        }
    })
}

/// Ollama-specific streaming handler with XML tool call fallback.
/// Uses the Ollama format module which buffers text when XML tool calls are detected,
/// preventing duplicate content from being emitted to the UI.
/// Timeout is applied at the raw SSE line level via with_line_timeout so that
/// buffering inside response_to_streaming_message_ollama does not cause false stalls.

View on GitHub (pinned to 3810898a74)

Solutions

  1. Raise the budget: export OLLAMA_STREAM_TIMEOUT=600 (checked first), or GOOSE_STREAM_TIMEOUT / OLLAMA_TIMEOUT; note OLLAMA_TIMEOUT also caps the overall request, so raise it too for long generations
  2. Shrink the payload: disable unneeded MCP tools/extensions, trim conversation history, lower num_ctx/context limit
  3. Use a smaller/quantized model or offload less (check 'ollama ps'); for CPU inference, budget minutes per response, not seconds
  4. Check ollama's own logs (journalctl -u ollama or OLLAMA_DEBUG=1) for OOM kills or model load errors — if the process died, no timeout increase will help
  5. If a proxy sits in the path, raise its read/idle timeout above OLLAMA_STREAM_TIMEOUT

Example fix

# before
export OLLAMA_TIMEOUT=600   # per-chunk gap still capped at 120s default -> stalls abort

# after
export OLLAMA_STREAM_TIMEOUT=600   # per-chunk watchdog
export OLLAMA_TIMEOUT=900           # overall request budget
# plus: goose session --no-extensions (or disable unused tools) to shrink the payload
Defensive patterns

Strategy: retry

Validate before calling

fn chunk_timeout_secs() -> u64 {
    ["OLLAMA_STREAM_TIMEOUT", "GOOSE_STREAM_TIMEOUT", "OLLAMA_TIMEOUT"]
        .iter().find_map(|k| std::env::var(k).ok().and_then(|v| v.parse().ok()))
        .unwrap_or(120)
}
// Before long jobs: ensure the budget fits the backend, e.g. bump for CPU inference:
// if running_on_cpu() { std::env::set_var("OLLAMA_STREAM_TIMEOUT", "600"); }

Try / catch

// Distinguish a stall (retryable, payload-driven) from a dead server (not retryable):
let mut attempt = 0;
loop {
    attempt += 1;
    match run_stream(&req).await {
        Err(e) if e.to_string().contains("stream stalled") && attempt < 3 => {
            tracing::warn!("stall on attempt {attempt}; trimming tools and retrying");
            req = trim_tools(req, 0.5).await; // shrink payload between attempts
            continue;
        }
        other => break other,
    }
}?;

Prevention

When it happens

Trigger: Streaming a large prompt (dozens of MCP tools, long context) to a slow backend — CPU-only inference, big parameter counts, deep reasoning models pausing between reasoning and output — where one inter-chunk gap exceeds the timeout; or the ollama daemon being OOM-killed / the connection severed mid-stream.

Common situations: Laptops running 70B models on CPU; goose sessions with many enabled MCP extensions so the tool schema bloats every request; reverse proxies (nginx/Cloudflare) with idle read timeouts shorter than the model's think time; ollama loading a second model and swapping VRAM mid-response.

Understand the failure class

Related errors

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/e2d2d789cf788270. Report an issue: GitHub.