{"record":{"id":"72be2a1a923f6921","repo":"invoke-ai/InvokeAI","slug":"text-generation-stalled-no-output-for-stream-tim","errorCode":null,"errorMessage":"Text generation stalled (no output for {STREAM_TIMEOUT}s)","messagePattern":"Text generation stalled \\(no output for (.+?)s\\)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/text_llm_pipeline.py","lineNumber":246,"sourceCode":"            if progress_callback is not None:\n                progress_callback(min(token_count, max_new_tokens), max_new_tokens)\n\n        try:\n            for chunk in streamer:\n                if not chunk:\n                    continue\n                chunks.append(chunk)\n                now = time.monotonic()\n                if progress_callback is not None and now - last_emit >= PROGRESS_EMIT_INTERVAL:\n                    _emit_progress()\n                    last_emit = now\n        except queue.Empty as e:\n            # The streamer timed out waiting for the next token: generate() stalled\n            # without raising and without signalling end(). Surface any captured error,\n            # otherwise raise a timeout rather than block on thread.join() below.\n            if generation_error:\n                raise generation_error[0] from e\n            raise RuntimeError(f\"Text generation stalled (no output for {STREAM_TIMEOUT}s)\") from e\n\n        # Guarantee a final emission so the reported token count is exact even if the\n        # last increment was throttled.\n        if progress_callback is not None and chunks:\n            _emit_progress()\n\n        thread.join()\n        if generation_error:\n            raise generation_error[0]\n\n        return \"\".join(chunks).strip()\n","sourceCodeStart":228,"sourceCodeEnd":258,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/text_llm_pipeline.py#L228-L258","documentation":"This error is raised by the text-generation streaming loop when the token streamer produced no output for STREAM_TIMEOUT seconds. It means the worker thread running generate() stalled: it neither emitted a token nor signalled end(). Any exception captured from the worker thread is re-raised first; this error only appears when the thread stalled silently.","triggerScenarios":"Calling the LLM pipeline's run() and the queue.get() in the streaming loop times out with queue.Empty because the underlying generate() call hangs (e.g. deadlocked model, GPU kernel hang, or the thread crashed without setting generation_error).","commonSituations":"Slow/hung GPU drivers, model loaded on a device that froze, an infinite retry inside generate(), or a misconfigured streamer whose end() is never called on completion.","solutions":["Check the model/GPU health — restart the worker or process if the device is wedged (nvidia-smi, dmesg)","Surface and fix any exception inside the worker's generate() that was swallowed without setting generation_error","Increase STREAM_TIMEOUT if generation is legitimately slow for very large prompts on slow hardware","Verify the streamer is correctly wired so end() is called when generation finishes"],"exampleFix":"// before\nrun(pipeline, prompt=\"...\", stream_timeout=STREAM_TIMEOUT)  # hangs on huge prompt on slow GPU\n// after\nrun(pipeline, prompt=\"...\", stream_timeout=120)  # raise timeout for legitimately slow generations","handlingStrategy":"try-catch","validationCode":"# before running, ensure streamer is wired and timeout suits hardware\nassert STREAM_TIMEOUT > expected_max_latency_seconds\n","typeGuard":null,"tryCatchPattern":"try:\n    result = run(pipeline, prompt=prompt, stream_timeout=STREAM_TIMEOUT)\nexcept RuntimeError as e:\n    if \"Text generation stalled\" in str(e):\n        log.error(\"LLM generation stalled; restarting worker thread\")\n        restart_worker()\n    else:\n        raise","preventionTips":["Set STREAM_TIMEOUT above realistic worst-case generation latency for your hardware","Ensure the worker thread always sets generation_error or calls end(), even on failure","Monitor GPU health (nvidia-smi, driver logs) for hangs","Add a watchdog that cancels and restarts stalled generations"],"tags":["timeout","streaming","llm","inference"],"backgroundTag":"stream-generation-timeout","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}