unslothai/unsloth · critical · RuntimeError

The inference worker stopped unexpectedly while loading the

Error message

The inference worker stopped unexpectedly while loading the model.

What it means

Raised while waiting for a model-load response: _read_resp timed out (returned None) and _ensure_subprocess_alive() reported the inference subprocess is gone. The message includes _subprocess_crash_message('wait') details — pid, signal (e.g. SIGKILL), exit code — which identify how the worker died during the load.

Source

Thrown at studio/backend/core/inference/orchestrator.py:584

        *timeout* is an **inactivity** timeout: it resets on each status
        message, so long-running operations (large downloads, slow loads)
        survive as long as the subprocess keeps reporting progress.
        """
        # Local: resolving this name runs the shim's lazy unsloth_zoo load, which pulls torch.
        # The shim caches its pick, so this site and load_model()'s `except` see one class.
        from utils.hf_xet_fallback import DownloadStallError

        deadline = time.monotonic() + timeout

        while time.monotonic() < deadline:
            remaining = max(0.1, deadline - time.monotonic())
            resp = self._read_resp(timeout = min(remaining, 1.0))

            if resp is None:
                # Check subprocess health
                if not self._ensure_subprocess_alive():
                    raise RuntimeError(self._subprocess_crash_message("wait"))
                continue

            rtype = resp.get("type", "")

            if rtype == expected_type:
                return resp

            if rtype == "error":
                error_msg = resp.get("error", "Unknown error")
                raise RuntimeError(f"Subprocess error: {error_msg}")

            if rtype == "status":
                logger.info("Subprocess status: %s", resp.get("message", ""))
                # Reset deadline — subprocess is still alive and working
                deadline = time.monotonic() + timeout
                continue

            if rtype == "stall":

View on GitHub (pinned to 203007d190)

Solutions

Read the signal/exitcode in the message: SIGKILL/-9 → reduce model size, enable offloading, or add RAM/swap; SIGSEGV/abort → check native deps and CUDA driver
Check dmesg/journal for OOM-killer entries at the crash timestamp
Reinstall/repair the inference sidecar if a version change correlates with the crash
Retry the load after fixing resources; loads are restartable

Example fix

# before
await orchestrator.load_model('llama-70b')  # worker OOM-killed, opaque failure

# after
await orchestrator.load_model('llama-70b', {
    'device_map': 'auto',
    'load_in_4bit': True,   # cut resident memory so the worker survives load
})

Defensive patterns

Strategy: try-catch

Validate before calling

import psutil

def enough_memory_for(model_bytes: int, path: str = None) -> bool:
    avail = psutil.virtual_memory().available
    return avail > int(model_bytes * 1.25)  # headroom for load spike

Try / catch

try:
    await orchestrator.load_model(model_id)
except RuntimeError as exc:
    if 'stopped unexpectedly while loading' in str(exc):
        report_crash_details(str(exc))     # pid/signal/exitcode for the user
        if 'signal=SIGKILL' in str(exc) or 'signal=9' in str(exc):
            suggest_smaller_model_or_more_ram()
        raise

Prevention

Pre-check available RAM against model size (with load headroom) before starting a load
Use quantization/offloading for models near the memory ceiling
Watch dmesg for OOM-killer events and alert on repeated worker deaths during load

When it happens

Trigger: load_model in flight; worker exits before sending the expected response type. The loop reads None (no message within its slice), checks liveness, gets False, and raises with the crash diagnostics.

Common situations: Out-of-memory kill (SIGKILL, exitcode -9) when loading a model larger than available RAM; missing native libraries causing abort during model import; CUDA/driver errors killing the process; stack/ABI mismatch after a sidecar version change.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/ba9abc3e88a86c4c. Report an issue: GitHub.