{"record":{"id":"2a2b4b288b0b4765","repo":"unslothai/unsloth","slug":"sd-server-is-not-running","errorCode":null,"errorMessage":"sd-server is not running.","messagePattern":"sd-server is not running\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/sd_cpp_server.py","lineNumber":453,"sourceCode":"        *,\n        on_step: Optional[Callable[[str], None]] = None,\n        cancel_event: Optional[threading.Event] = None,\n        poll_interval: float = 0.4,\n        submit_timeout: float = 60.0,\n        total_timeout: float = NATIVE_GENERATION_TIMEOUT_S,\n    ) -> list[bytes]:\n        \"\"\"Submit one async ``img_gen`` job, poll it to completion, return image bytes.\n\n        ``on_step`` receives each server stdout line (for the step bar). ``cancel_event``,\n        when set, cancels the job via the native endpoint and raises ``SdCppCancelled``.\n        Raises ``RuntimeError`` on submit/poll failures (including the server dying), with\n        the log tail attached.\n        \"\"\"\n        # Already stopped with the cancel event set: report cancellation (route 409), not a generic \"server died\" 500.\n        if self._stopped or not self.is_alive():\n            if cancel_event is not None and cancel_event.is_set():\n                raise SdCppCancelled(\"sd-server generation was cancelled.\")\n            raise RuntimeError(\"sd-server is not running.\")\n\n        self._step_listener = on_step\n        job_id: Optional[str] = None\n        try:\n            # Submit -> 202 Accepted + job id.\n            try:\n                resp = self._client.post(\n                    f\"{self.base_url}{_IMG_GEN_PATH}\", json = payload, timeout = submit_timeout\n                )\n            except (*_TRANSPORT_ERRORS, httpx.TimeoutException) as exc:\n                raise RuntimeError(self._died_message(\"img_gen submit\", exc)) from exc\n            if resp.status_code == 429:\n                raise RuntimeError(\"sd-server job queue is full (HTTP 429).\")\n            if resp.status_code not in (200, 202):\n                raise RuntimeError(\n                    f\"sd-server img_gen submit -> {resp.status_code}: {resp.text[:500]}\"\n                )\n            try:","sourceCodeStart":435,"sourceCodeEnd":471,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/sd_cpp_server.py#L435-L471","documentation":"Raised at the top of img_gen when the wrapper sees itself stopped or its child process not alive, and the cancel event is NOT set — i.e. the server died or was shut down for a non-cancellation reason. The check deliberately routes 'stopped + cancelled' to SdCppCancelled (409-style) so users see 'cancelled', while everything else becomes this generic not-running error. Companion diagnostics (server died mid-job) come from _died_message elsewhere.","triggerScenarios":"Calling img_gen after the server crashed from a previous job, after an explicit stop()/dispose(), or after the child exited (OOM-killed, watchdog-killed).","commonSituations":"Server killed by the OS OOM killer during an earlier large batch; lifecycle code disposing the server while an async UI action still submits a job; stale wrapper reference after a restart.","solutions":["Restart the server (reload/re-start the wrapper) and resubmit the job.","Check system logs (dmesg / journalctl) for OOM-kill of the sd-server pid to find the root cause of death.","Audit lifecycle ordering: never stop/dispose the server while jobs can still be submitted from other threads."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"if server.is_stopped() or not server.is_alive():\n    server = restart_server()  # before submitting","typeGuard":"def server_ready(server) -> bool:\n    return (not server._stopped) and server.is_alive()","tryCatchPattern":"try:\n    blobs = server.img_gen(payload, ...)\nexcept RuntimeError as e:\n    if \"not running\" in str(e):\n        server = restart_server()\n        blobs = server.img_gen(payload, ...)\n    raise","preventionTips":["Check server liveness before submitting jobs from async contexts.","Never dispose the server while submit-capable threads can still run.","Monitor the server pid for OOM kills if deaths recur."],"tags":["diffusion","sd-cpp","server","lifecycle"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}