stream timed out due to inactivity. The model is not respond

Error message

stream timed out due to inactivity. The model is not responding.

What it means

The stream's inactivity watchdog timer fired while the stream was still open: no chunk arrived before the timeout. The library returns the accumulated partial result with this error attached (Result(true, err)) instead of propagating a hard failure. It means the model/provider accepted the request but stopped sending data.

Source

Thrown at app/server/model/client_stream.go:134

	defer timer.Stop()
	streamFinished := false

	receivedFirstChunk := false

	// Process stream until EOF or error
	for {
		select {
		case <-streamCtx.Done():
			log.Println("Stream canceled")
			return accumulator.Result(true, streamCtx.Err()), streamCtx.Err()
		case <-timer.C:
			log.Println("Stream timed out due to inactivity")
			if streamFinished {
				log.Println("Stream finished—timed out waiting for usage chunk")
				return accumulator.Result(false, nil), nil
			} else {
				log.Println("Stream timed out due to inactivity")
				return accumulator.Result(true, fmt.Errorf("stream timed out due to inactivity. The model is not responding.")), nil
			}
		default:
			response, err := stream.Recv()
			if err == io.EOF {
				if streamFinished {
					return accumulator.Result(false, nil), nil
				}

				err = fmt.Errorf("model stream ended unexpectedly: %w", err)
				return accumulator.Result(true, err), err
			}
			if err != nil {
				err = fmt.Errorf("error receiving stream chunk: %w", err)
				return accumulator.Result(true, err), err
			}

			if response.ID != "" {
				accumulator.SetGenerationId(response.ID)

View on GitHub (pinned to e2d772072e)

Solutions

Retry the request, possibly with a shorter prompt or lower max_tokens
Increase the inactivity timeout if the model legitimately takes long before first/next token
Check provider status page or proxy health (isLiteLLMHealthy) for outages
Verify the chosen model is actually producing output (test with a trivial prompt)
Treat the returned partial accumulator.Result(true, err) content as usable-or-discard depending on product needs

Example fix

// before: single attempt dies on idle timeout
result, err := processChatCompletionStream(...)
// after: retry once on inactivity timeout
result, err := processChatCompletionStream(...)
if err != nil && strings.Contains(err.Error(), "timed out due to inactivity") {
    result, err = processChatCompletionStream(...) // retry
}

Defensive patterns

Strategy: retry

Type guard

func isInactivityTimeout(err error) bool { return err != nil && strings.Contains(err.Error(), "timed out due to inactivity") }

Try / catch

result, err := call()
if isInactivityTimeout(err) && attempt < maxAttempts {
    time.Sleep(backoff)
    return call()
}

Prevention

Set an inactivity timeout appropriate to the model's worst-case latency
Monitor provider status and configure fallback models
Avoid routing long streams through proxies with short idle timeouts
Alert on recurring inactivity timeouts per model to catch degradations

When it happens

Trigger: stream.Recv() has not returned a chunk before the inactivity timer deadline and streamFinished is false; the select's timer case fires before a Recv result or EOF.

Common situations: Provider outage or brownout mid-request; extremely long model 'thinking' time exceeding the inactivity timeout; hung proxy (e.g., LiteLLM) between client and provider; connection silently dropped without TCP close.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of plandex-ai/plandex@e2d772072e (2026-09-05). Data as JSON: /api/errors/c24da64a114a92ef. Report an issue: GitHub.