{"record":{"id":"fbc200d279f9d3e0","repo":"unslothai/unsloth","slug":"sd-server-failed-to-become-ready-last-output-ta","errorCode":null,"errorMessage":"sd-server failed to become ready. Last output:\n{tail[:2000]}","messagePattern":"sd-server failed to become ready\\. Last output:\n(.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/sd_cpp_server.py","lineNumber":284,"sourceCode":"                except Exception:  # noqa: BLE001\n                    pass\n\n            self._stdout_thread = threading.Thread(\n                target = _own_process, daemon = True, name = \"sd-server-owner\"\n            )\n            self._stdout_thread.start()\n            spawned.wait()\n            if self._spawn_error is not None:\n                self._dispose()\n                raise RuntimeError(f\"failed to spawn sd-server: {self._spawn_error}\")\n            if not self._wait_ready(startup_timeout):\n                tail = _diagnostic_tail(self._tail, keep = 30)\n                aborted = self._abort.is_set()\n                self._kill_locked()\n                self._dispose()\n                if aborted:\n                    raise SdCppCancelled(\"sd-server startup was cancelled.\")\n                raise RuntimeError(\"sd-server failed to become ready. Last output:\\n\" + tail[:2000])\n\n    def _wait_ready(\n        self,\n        timeout: float,\n        interval: float = 0.5,\n    ) -> bool:\n        \"\"\"Poll ``/v1/models`` until 200; bail early if the process exits.\n\n        Upstream binds the port only AFTER the model is loaded, so a 200 here is a true\n        ready signal (no half-loaded race).\"\"\"\n        deadline = time.monotonic() + timeout\n        url = f\"{self.base_url}{_READY_PATH}\"\n        while time.monotonic() < deadline:\n            # A concurrent stop() sets _abort so this wait bails without holding the model load hostage for the full startup_timeout.\n            if self._abort.is_set():\n                logger.info(\"sd-server startup aborted before ready\")\n                return False\n            if not self.is_alive():","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/sd_cpp_server.py#L266-L302","documentation":"The sd-server process spawned but did not answer /v1/models with HTTP 200 within startup_timeout. Upstream binds the port only after the model finishes loading, so this almost always means model load is slow (large model, cold page cache, slow disk) or the server crashed during load. The wrapper attaches the last 30 log lines (truncated to 2000 chars), kills the process, and disposes it; if the abort event was set it raises SdCppCancelled instead.","triggerScenarios":"Server.start() with a startup_timeout shorter than the model load time; model file corruption making load fail after bind-less startup; OOM while loading weights.","commonSituations":"First load of a 70B-class or heavily quantized diffusion model on a slow disk; timeout left at a small default after switching to a much bigger model; cold cache after reboot.","solutions":["Pass a larger startup_timeout proportional to model size and disk speed.","Read the embedded tail: load-stage crashes (OOM, corrupt file) show up there — fix those rather than raising the timeout.","Prewarm the page cache (one manual load) or move the model to faster storage to cut load time.","Re-download a model whose load consistently dies with checksum/parse errors in the tail."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"size_gb = model_path.stat().st_size / 2**30\ntimeout = max(startup_timeout, 60 + size_gb * 8)  # scale with model size","typeGuard":null,"tryCatchPattern":"try:\n    server.start(startup_timeout=t)\nexcept RuntimeError as e:\n    if \"failed to become ready\" in str(e):\n        triage_tail(e); server.start(startup_timeout=t * 3)\n    raise","preventionTips":["Scale startup_timeout with model size and disk speed.","Read the embedded tail before retrying — load crashes need a fix, not more time.","Keep models on fast local storage and prewarm the cache for huge models."],"tags":["diffusion","sd-cpp","server","startup","timeout"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}