unslothai/unsloth · error · RuntimeError

sd-server is not running.

Error message

sd-server is not running.

What it means

Raised at the top of img_gen when the wrapper sees itself stopped or its child process not alive, and the cancel event is NOT set — i.e. the server died or was shut down for a non-cancellation reason. The check deliberately routes 'stopped + cancelled' to SdCppCancelled (409-style) so users see 'cancelled', while everything else becomes this generic not-running error. Companion diagnostics (server died mid-job) come from _died_message elsewhere.

Source

Thrown at studio/backend/core/inference/sd_cpp_server.py:453

        *,
        on_step: Optional[Callable[[str], None]] = None,
        cancel_event: Optional[threading.Event] = None,
        poll_interval: float = 0.4,
        submit_timeout: float = 60.0,
        total_timeout: float = NATIVE_GENERATION_TIMEOUT_S,
    ) -> list[bytes]:
        """Submit one async ``img_gen`` job, poll it to completion, return image bytes.

        ``on_step`` receives each server stdout line (for the step bar). ``cancel_event``,
        when set, cancels the job via the native endpoint and raises ``SdCppCancelled``.
        Raises ``RuntimeError`` on submit/poll failures (including the server dying), with
        the log tail attached.
        """
        # Already stopped with the cancel event set: report cancellation (route 409), not a generic "server died" 500.
        if self._stopped or not self.is_alive():
            if cancel_event is not None and cancel_event.is_set():
                raise SdCppCancelled("sd-server generation was cancelled.")
            raise RuntimeError("sd-server is not running.")

        self._step_listener = on_step
        job_id: Optional[str] = None
        try:
            # Submit -> 202 Accepted + job id.
            try:
                resp = self._client.post(
                    f"{self.base_url}{_IMG_GEN_PATH}", json = payload, timeout = submit_timeout
                )
            except (*_TRANSPORT_ERRORS, httpx.TimeoutException) as exc:
                raise RuntimeError(self._died_message("img_gen submit", exc)) from exc
            if resp.status_code == 429:
                raise RuntimeError("sd-server job queue is full (HTTP 429).")
            if resp.status_code not in (200, 202):
                raise RuntimeError(
                    f"sd-server img_gen submit -> {resp.status_code}: {resp.text[:500]}"
                )
            try:

View on GitHub (pinned to 203007d190)

Solutions

1. Restart the server (reload/re-start the wrapper) and resubmit the job.
2. Check system logs (dmesg / journalctl) for OOM-kill of the sd-server pid to find the root cause of death.
3. Audit lifecycle ordering: never stop/dispose the server while jobs can still be submitted from other threads.

if server.is_stopped() or not server.is_alive():
    server = restart_server()  # before submitting

def server_ready(server) -> bool:
    return (not server._stopped) and server.is_alive()

try:
    blobs = server.img_gen(payload, ...)
except RuntimeError as e:
    if "not running" in str(e):
        server = restart_server()
        blobs = server.img_gen(payload, ...)
    raise

When it happens

Trigger: Calling img_gen after the server crashed from a previous job, after an explicit stop()/dispose(), or after the child exited (OOM-killed, watchdog-killed).

Common situations: Server killed by the OS OOM killer during an earlier large batch; lifecycle code disposing the server while an async UI action still submits a job; stale wrapper reference after a restart.

Related errors

• failed to spawn sd-server: {self._spawn_error}
• sd-server failed to become ready. Last output: {tail[:2000]}
• sd-server connection lost during img_gen submit
• sd-server job queue is full (HTTP 429).
• sd-server img_gen submit -> {resp.status_code}: {resp.text[:

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/2a2b4b288b0b4765. Report an issue: GitHub.