{"record":{"id":"ba9abc3e88a86c4c","repo":"unslothai/unsloth","slug":"the-inference-worker-stopped-unexpectedly-while-lo","errorCode":null,"errorMessage":"The inference worker stopped unexpectedly while loading the model.","messagePattern":"The inference worker stopped unexpectedly while loading the model\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":500,"severity":"critical","filePath":"studio/backend/core/inference/orchestrator.py","lineNumber":584,"sourceCode":"\n        *timeout* is an **inactivity** timeout: it resets on each status\n        message, so long-running operations (large downloads, slow loads)\n        survive as long as the subprocess keeps reporting progress.\n        \"\"\"\n        # Local: resolving this name runs the shim's lazy unsloth_zoo load, which pulls torch.\n        # The shim caches its pick, so this site and load_model()'s `except` see one class.\n        from utils.hf_xet_fallback import DownloadStallError\n\n        deadline = time.monotonic() + timeout\n\n        while time.monotonic() < deadline:\n            remaining = max(0.1, deadline - time.monotonic())\n            resp = self._read_resp(timeout = min(remaining, 1.0))\n\n            if resp is None:\n                # Check subprocess health\n                if not self._ensure_subprocess_alive():\n                    raise RuntimeError(self._subprocess_crash_message(\"wait\"))\n                continue\n\n            rtype = resp.get(\"type\", \"\")\n\n            if rtype == expected_type:\n                return resp\n\n            if rtype == \"error\":\n                error_msg = resp.get(\"error\", \"Unknown error\")\n                raise RuntimeError(f\"Subprocess error: {error_msg}\")\n\n            if rtype == \"status\":\n                logger.info(\"Subprocess status: %s\", resp.get(\"message\", \"\"))\n                # Reset deadline — subprocess is still alive and working\n                deadline = time.monotonic() + timeout\n                continue\n\n            if rtype == \"stall\":","sourceCodeStart":566,"sourceCodeEnd":602,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/orchestrator.py#L566-L602","documentation":"Raised while waiting for a model-load response: _read_resp timed out (returned None) and _ensure_subprocess_alive() reported the inference subprocess is gone. The message includes _subprocess_crash_message('wait') details — pid, signal (e.g. SIGKILL), exit code — which identify how the worker died during the load.","triggerScenarios":"load_model in flight; worker exits before sending the expected response type. The loop reads None (no message within its slice), checks liveness, gets False, and raises with the crash diagnostics.","commonSituations":"Out-of-memory kill (SIGKILL, exitcode -9) when loading a model larger than available RAM; missing native libraries causing abort during model import; CUDA/driver errors killing the process; stack/ABI mismatch after a sidecar version change.","solutions":["Read the signal/exitcode in the message: SIGKILL/-9 → reduce model size, enable offloading, or add RAM/swap; SIGSEGV/abort → check native deps and CUDA driver","Check dmesg/journal for OOM-killer entries at the crash timestamp","Reinstall/repair the inference sidecar if a version change correlates with the crash","Retry the load after fixing resources; loads are restartable"],"exampleFix":"# before\nawait orchestrator.load_model('llama-70b')  # worker OOM-killed, opaque failure\n\n# after\nawait orchestrator.load_model('llama-70b', {\n    'device_map': 'auto',\n    'load_in_4bit': True,   # cut resident memory so the worker survives load\n})","handlingStrategy":"try-catch","validationCode":"import psutil\n\ndef enough_memory_for(model_bytes: int, path: str = None) -> bool:\n    avail = psutil.virtual_memory().available\n    return avail > int(model_bytes * 1.25)  # headroom for load spike","typeGuard":null,"tryCatchPattern":"try:\n    await orchestrator.load_model(model_id)\nexcept RuntimeError as exc:\n    if 'stopped unexpectedly while loading' in str(exc):\n        report_crash_details(str(exc))     # pid/signal/exitcode for the user\n        if 'signal=SIGKILL' in str(exc) or 'signal=9' in str(exc):\n            suggest_smaller_model_or_more_ram()\n        raise","preventionTips":["Pre-check available RAM against model size (with load headroom) before starting a load","Use quantization/offloading for models near the memory ceiling","Watch dmesg for OOM-killer events and alert on repeated worker deaths during load"],"tags":["orchestrator","subprocess","oom","model-loading","crash"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}