{"record":{"id":"11a4f91ad729d0a4","repo":"unslothai/unsloth","slug":"dictation-model-loading-was-cancelled-so-training","errorCode":null,"errorMessage":"Dictation model loading was cancelled so training could start.","messagePattern":"Dictation model loading was cancelled so training could start\\.","errorType":"http","errorClass":"SttLoadCancelledError","httpStatus":409,"severity":"info","filePath":"studio/backend/core/inference/stt_mtmd_sidecar.py","lineNumber":924,"sourceCode":"                # Bundled libs and pip CUDA runtimes on the loader path, secrets\n                # scrubbed, as the chat backend spawns the same binary.\n                env = _llama_server_child_env(binary),\n                # Die with Studio, so a crash never orphans a server on the GPU.\n                **child_popen_kwargs(),\n            )\n            # Published before the wait, so training can preempt a startup that\n            # is already allocating; _process is not set for another 180s.\n            with self._lock:\n                self._starting_process = process\n            adopt_pid(process.pid)  # terminate_all backstop for graceful exits\n            if not self._wait_for_server(process, port, cancel_event):\n                # Reap it here: _process was never assigned, so unload() cannot\n                # reach a child that ignores SIGTERM and keeps port and VRAM.\n                _reap(process)\n                if cancel_event.is_set():\n                    # 409 through the route, like the other sidecars: expected\n                    # preemption, not a broken or missing runtime (501).\n                    raise SttLoadCancelledError(\n                        \"Dictation model loading was cancelled so training could start.\"\n                    )\n                raise SttUnavailableError(f\"llama-server did not become ready for '{model_id}'.\")\n            with self._lock:\n                self._process = process\n                self._port = port\n                self._model_id = model_id\n                self._gpu_disabled = training\n                self._generation += 1\n                self._schedule_idle_unload_locked()\n        finally:\n            with self._lock:\n                self._loading = False\n                self._load_cancel_event = None\n                self._load_owner_cancel_event = None\n                self._starting_process = None\n\n    @staticmethod","sourceCodeStart":906,"sourceCodeEnd":942,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/stt_mtmd_sidecar.py#L906-L942","documentation":"SttLoadCancelledError raised when _wait_for_server() returns False and the cancel event is set: the freshly spawned llama-server never became ready and was cancelled (typically preempted by a training run that needs the VRAM). The half-started process is reaped before raising so no port or VRAM leaks.","triggerScenarios":"A training run (or explicit cancel) sets the load's cancel event while llama-server is still in its up-to-180s startup window — the readiness wait is aborted.","commonSituations":"User starts a fine-tune right after beginning to dictate; GPU is slow to allocate and the wait is preempted; keep-alive preload cancelled by a scheduled training job.","solutions":["Treat as expected preemption (surfaces as 409): let the user re-run dictation after training starts or completes.","If training and dictation collide often, preload the dictation model before launching training runs.","If it fires with no training active, check that nothing else is setting the shared cancel event."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"```python\ntry:\n    sidecar.load(model_id)\nexcept SttLoadCancelledError:\n    return {\"status\": \"preempted-by-training\"}, 409  # retry after training ends\n```","preventionTips":["Preload the dictation model before launching training runs.","Treat 409 preemption as expected; offer the user a one-click retry.","Do not alert on this class — it is designed behavior, not a runtime fault."],"tags":["stt","cancellation","startup-timeout","training-preemption","vram"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}