invoke-ai/InvokeAI · error · RuntimeError

Text generation stalled (no output for {STREAM_TIMEOUT}s)

Error message

Text generation stalled (no output for {STREAM_TIMEOUT}s)

What it means

This error is raised by the text-generation streaming loop when the token streamer produced no output for STREAM_TIMEOUT seconds. It means the worker thread running generate() stalled: it neither emitted a token nor signalled end(). Any exception captured from the worker thread is re-raised first; this error only appears when the thread stalled silently.

Source

Thrown at invokeai/backend/text_llm_pipeline.py:246

            if progress_callback is not None:
                progress_callback(min(token_count, max_new_tokens), max_new_tokens)

        try:
            for chunk in streamer:
                if not chunk:
                    continue
                chunks.append(chunk)
                now = time.monotonic()
                if progress_callback is not None and now - last_emit >= PROGRESS_EMIT_INTERVAL:
                    _emit_progress()
                    last_emit = now
        except queue.Empty as e:
            # The streamer timed out waiting for the next token: generate() stalled
            # without raising and without signalling end(). Surface any captured error,
            # otherwise raise a timeout rather than block on thread.join() below.
            if generation_error:
                raise generation_error[0] from e
            raise RuntimeError(f"Text generation stalled (no output for {STREAM_TIMEOUT}s)") from e

        # Guarantee a final emission so the reported token count is exact even if the
        # last increment was throttled.
        if progress_callback is not None and chunks:
            _emit_progress()

        thread.join()
        if generation_error:
            raise generation_error[0]

        return "".join(chunks).strip()

View on GitHub (pinned to 0b6a024f2f)

Solutions

  1. Check the model/GPU health — restart the worker or process if the device is wedged (nvidia-smi, dmesg)
  2. Surface and fix any exception inside the worker's generate() that was swallowed without setting generation_error
  3. Increase STREAM_TIMEOUT if generation is legitimately slow for very large prompts on slow hardware
  4. Verify the streamer is correctly wired so end() is called when generation finishes

Example fix

// before
run(pipeline, prompt="...", stream_timeout=STREAM_TIMEOUT)  # hangs on huge prompt on slow GPU
// after
run(pipeline, prompt="...", stream_timeout=120)  # raise timeout for legitimately slow generations
Defensive patterns

Strategy: try-catch

Validate before calling

# before running, ensure streamer is wired and timeout suits hardware
assert STREAM_TIMEOUT > expected_max_latency_seconds

Try / catch

try:
    result = run(pipeline, prompt=prompt, stream_timeout=STREAM_TIMEOUT)
except RuntimeError as e:
    if "Text generation stalled" in str(e):
        log.error("LLM generation stalled; restarting worker thread")
        restart_worker()
    else:
        raise

Prevention

When it happens

Trigger: Calling the LLM pipeline's run() and the queue.get() in the streaming loop times out with queue.Empty because the underlying generate() call hangs (e.g. deadlocked model, GPU kernel hang, or the thread crashed without setting generation_error).

Common situations: Slow/hung GPU drivers, model loaded on a device that froze, an infinite retry inside generate(), or a misconfigured streamer whose end() is never called on completion.

Understand the failure class

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/72be2a1a923f6921. Report an issue: GitHub.