unslothai/unsloth · error · RuntimeError

No inference subprocess running

Error message

No inference subprocess running

What it means

Orchestrator._send_cmd raises this RuntimeError when it is asked to enqueue a command but self._cmd_queue is None — the inference subprocess (and its queues) was never started or has already been torn down. Every control command (load, generate, stop) goes through this queue, so any command sent outside a subprocess's lifetime fails here.

Source

Thrown at studio/backend/core/inference/orchestrator.py:540

            suffix = ""
            if sig_name == "SIGKILL":
                suffix = (
                    " This usually means the system killed it under memory pressure. "
                    "Try a smaller model, lower context length, or close other GPU-heavy apps."
                )
            return f"{message}{suffix} Details: pid={pid}, signal={sig_name}, exitcode={exitcode}."

        return f"{message} Details: pid={pid}, exitcode={exitcode}."

    # ------------------------------------------------------------------
    # Queue helpers
    # ------------------------------------------------------------------

    def _send_cmd(self, cmd: dict) -> None:
        """Send a command to the subprocess."""
        if self._cmd_queue is None:
            raise RuntimeError("No inference subprocess running")
        try:
            self._cmd_queue.put(cmd)
        except (OSError, ValueError) as exc:
            raise RuntimeError(f"Failed to send command to subprocess: {exc}")

    def _read_resp(self, timeout: float = 1.0) -> Optional[dict]:
        """Read a response from the subprocess (non-blocking with timeout)."""
        if self._resp_queue is None:
            return None
        try:
            return self._resp_queue.get(timeout = timeout)
        except queue.Empty:
            return None
        except (EOFError, OSError, ValueError):
            return None

    def _wait_response(
        self,

View on GitHub (pinned to 203007d190)

Solutions

1. Check the subprocess is alive (e.g. _ensure_subprocess_alive / is_running equivalent) before issuing commands
2. Start the subprocess before the first command in the request path
3. On this error, restart the subprocess and re-issue the command rather than propagating to the user

Example fix

# before
orchestrator._send_cmd({'type': 'generate', ...})

# after
if not orchestrator.is_running():
    await orchestrator.start_subprocess()
orchestrator._send_cmd({'type': 'generate', ...})

if not orchestrator.is_running():  # or: orchestrator._cmd_queue is None
    await orchestrator.start_subprocess()

def can_send_commands(orchestrator) -> bool:
    return orchestrator._cmd_queue is not None

try:
    orchestrator._send_cmd(cmd)
except RuntimeError as exc:
    if 'No inference subprocess running' in str(exc):
        await orchestrator.start_subprocess()
        orchestrator._send_cmd(cmd)
    else:
        raise

When it happens

Trigger: Calling any command-sending method (e.g. load/generate/stop paths that use _send_cmd) before start_subprocess() created the queues, or after stop()/crash teardown set them to None.

Common situations: Code path that generates without checking the worker is running; race where the worker crashed and teardown finished between a liveness check and the command send; calling orchestrator methods after explicit shutdown.

Related errors

• No export subprocess running
• Failed to send command to subprocess: {exc}
• The inference worker stopped unexpectedly while loading the
• Failed to send command to subprocess: {exc}
• Export subprocess crashed during wait

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/72d35b902b63131c. Report an issue: GitHub.