unslothai/unsloth · error · RuntimeError

Inference subprocess is not running

Error message

Inference subprocess is not running

What it means

Guard at the top of the TTS generate_audio path: _ensure_subprocess_alive() returned False, meaning there is no running inference worker (never started, crashed, or shut down). No audio request can even be enqueued in this state.

Source

Thrown at studio/backend/core/inference/orchestrator.py:2111

    def generate_audio_response(
        self,
        text: str,
        temperature: float = 0.6,
        top_p: float = 0.95,
        top_k: int = 50,
        min_p: float = 0.0,
        max_new_tokens: int = 2048,
        repetition_penalty: float = 1.0,
        use_adapter: Optional[Union[bool, str]] = None,
        cancel_event = None,
    ) -> Tuple[bytes, int]:
        """Generate TTS audio. Returns (wav_bytes, sample_rate).

        Blocking — sends command and waits for the full audio response.
        """
        if not self._ensure_subprocess_alive():
            raise RuntimeError("Inference subprocess is not running")
        if not self.active_model_name:
            raise RuntimeError("No active model")
        expected_model = self.active_model_name

        # Serialize under _gen_lock and reserve dispatcher admission before waiting for
        # compare work to drain. A bare idle wait is racy: a compare request can register
        # between the wait and this command, leaving TTS queued without safe ownership of
        # the worker's single shared cancel event.
        with self._gen_lock:
            with self._dispatcher_lifecycle_lock:
                self._exclusive_tts_pending = True
            try:
                dispatcher_idle = self._wait_dispatcher_idle(cancel_event = cancel_event)
                if cancel_event is not None and cancel_event.is_set():
                    raise AudioGenerationCancelledError("Audio generation cancelled")
                if not dispatcher_idle:
                    raise RuntimeError(
                        "Cannot start audio generation while compare requests are active"

View on GitHub (pinned to 203007d190)

Solutions

1. Load a TTS-capable model first and wait for it to finish before requesting audio.
2. Check backend logs for why the subprocess is gone (crash, stall teardown, cancel drain).
3. Retry after a successful model load re-spawns the worker.
4. If the worker keeps dying, fix the underlying crash (see the worker error in logs).

Example fix

// before
wav, sr = orchestrator.generate_audio(text)
// after
if not orchestrator.is_worker_alive():
    orchestrator.load_model(tts_model)
wav, sr = orchestrator.generate_audio(text)

if not orchestrator.is_worker_alive():
    orchestrator.load_model(tts_model)  # spawn worker before audio

def can_generate_audio(orch) -> bool:
    return orch.is_worker_alive() and bool(orch.active_model_name)

When it happens

Trigger: Calling generate_audio (TTS) before any model was loaded, after the subprocess died from an OOM/crash, or after a shutdown triggered by a prior cancel/stall teardown.

Common situations: Frontend fires a TTS request on app startup before load completes; worker crashed earlier in the session and the user retries audio; load was cancelled leaving no worker.

Related errors

• No active model
• Subprocess error: {error_msg}
• Timeout waiting for '{expected_type}' response (no activity
• The current inference worker did not exit and still holds GP
• Audio generation cancelled

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/38cca919f85e2f6f. Report an issue: GitHub.