{"record":{"id":"d8c06d044f8919f7","repo":"unslothai/unsloth","slug":"sd-cli-timed-out-after-timeout-s","errorCode":null,"errorMessage":"sd-cli timed out after {timeout}s","messagePattern":"sd-cli timed out after (.+?)s","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/sd_cpp_engine.py","lineNumber":795,"sourceCode":"                for rec in iter_sd_cpp_records(proc.stdout):\n                    line_q.put(rec)\n            finally:\n                line_q.put(None)\n\n        reader = threading.Thread(target = _drain, daemon = True)\n        reader.start()\n\n        deadline = None if timeout is None else time.monotonic() + float(timeout)\n        stdout_done = False\n        try:\n            while True:\n                # Cancellation: kill the process tree and signal cancelled, not failure.\n                if cancel_event is not None and cancel_event.is_set() and proc.poll() is None:\n                    _terminate(proc)\n                    raise SdCppCancelled(\"sd-cli generation was cancelled.\")\n                if deadline is not None and time.monotonic() >= deadline and proc.poll() is None:\n                    _terminate(proc)\n                    raise RuntimeError(f\"sd-cli timed out after {timeout}s\")\n                try:\n                    line = line_q.get(timeout = 0.1)\n                except queue.Empty:\n                    if proc.poll() is not None and stdout_done:\n                        break\n                    continue\n                if line is None:\n                    stdout_done = True\n                    if proc.poll() is not None:\n                        break\n                    continue\n                tail.append(line)\n                if len(tail) > 40:\n                    tail.pop(0)\n                if on_log is not None:\n                    on_log(line)\n            ret = proc.wait(timeout = 5.0)\n        finally:","sourceCodeStart":777,"sourceCodeEnd":813,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/sd_cpp_engine.py#L777-L813","documentation":"The sd-cli subprocess watchdog kills the process tree and raises this when the monotonic deadline (start + timeout) passes while the process is still running. The loop polls every 100ms and distinguishes timeout from cancellation (cancel raises SdCppCancelled instead). A None timeout disables the check entirely.","triggerScenarios":"Calling the CLI generate with a finite timeout smaller than the actual generation time — big resolutions, many steps, LoRA materialization, or CPU-only execution easily exceed tight defaults.","commonSituations":"Default timeout on slow hardware; steps/resolution raised without raising timeout; heavy machine load (another model training) starving the CLI of CPU/GPU.","solutions":["Pass a larger (or None) timeout to the generate call.","Reduce steps, resolution, or batch size so the run fits the budget.","Free the machine (close competing GPU/CPU jobs) or move to the server path, which streams progress and handles long runs better."],"exampleFix":"# before\nengine.generate(files, params, output_path=out, timeout=60)\n# after\nengine.generate(files, params, output_path=out, timeout=1800)","handlingStrategy":"retry","validationCode":"estimated = steps * (width * height) / 500_000  # rough seconds heuristic\ntimeout = max(timeout, estimated * 4)","typeGuard":null,"tryCatchPattern":"try:\n    engine.generate(files, params, output_path=out, timeout=t)\nexcept RuntimeError as e:\n    if \"timed out\" in str(e) and t is not None:\n        engine.generate(files, params, output_path=out, timeout=t * 3)\n    raise","preventionTips":["Scale timeout with steps x resolution, not a fixed constant.","Prefer the server path for long runs; it reports progress and cancels cleanly.","Pass timeout=None for unattended batch jobs you monitor externally."],"tags":["diffusion","sd-cpp","timeout","cli"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}