{"record":{"id":"6f084017120cc1c8","repo":"unslothai/unsloth","slug":"stt-model-loading-was-cancelled-so-training-could","errorCode":null,"errorMessage":"STT model loading was cancelled so training could start.","messagePattern":"STT model loading was cancelled so training could start\\.","errorType":"http","errorClass":"SttLoadCancelledError","httpStatus":409,"severity":"info","filePath":"studio/backend/core/inference/stt_sidecar.py","lineNumber":1229,"sourceCode":"    def _begin_load(self, owner: Optional[threading.Event] = None) -> threading.Event:\n        event = owner if owner is not None else threading.Event()\n        with self._load_state_lock:\n            self._load_cancel_event = event\n            self._load_owner_cancel_event = owner\n            self._loading = True\n        return event\n\n    def _end_load(self, event: threading.Event) -> None:\n        with self._load_state_lock:\n            if self._load_cancel_event is event:\n                self._load_cancel_event = None\n                self._load_owner_cancel_event = None\n                self._loading = False\n\n    @staticmethod\n    def _raise_if_load_cancelled(event: threading.Event) -> None:\n        if event.is_set():\n            raise SttLoadCancelledError(\"STT model loading was cancelled so training could start.\")\n\n    @property\n    def keep_alive_seconds(self) -> float:\n        return self._keep_alive_seconds\n\n    def _cancel_idle_unload_locked(self) -> None:\n        self._idle_generation += 1\n        timer = self._idle_timer\n        self._idle_timer = None\n        if timer is not None:\n            timer.cancel()\n\n    def _schedule_idle_unload_locked(self) -> None:\n        self._cancel_idle_unload_locked()\n        if self._engine is None or self._keep_alive_seconds <= 0:\n            return\n        generation = self._idle_generation\n        timer = threading.Timer(","sourceCodeStart":1211,"sourceCodeEnd":1247,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/stt_sidecar.py#L1211-L1247","documentation":"Raised by _raise_if_load_cancelled when the load-cancel event for an in-progress STT model load is set. Its purpose, per the message, is to abort model loading so GPU/CPU memory is freed for training to start — load paths check the event at defined checkpoints.","triggerScenarios":"A training job requests GPU while load() is mid-flight: the training scheduler sets the sidecar's load cancel event, and the next checkpoint inside load (after download check, after _pick_device, around model build) raises SttLoadCancelledError.","commonSituations":"Studio apps where dictation and training share one GPU; user starts training while a Whisper model is still loading; preemption logic that cancels STT loads whenever a training run is admitted.","solutions":["Nothing is wrong: catch SttLoadCancelledError and retry load() after training starts or finishes, since memory priority resolved the conflict.","If loads are cancelled constantly, schedule STT loads when no training run is queued instead of racing them.","Give the sidecar its own device/resource budget so training admission does not need to cancel loads."],"exampleFix":"# before\nengine = sidecar.load(model)  # raises SttLoadCancelledError under training\n# after\ntry:\n    engine = sidecar.load(model)\nexcept SttLoadCancelledError:\n    schedule_retry_after_training(); return","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    engine = sidecar.load(model)\nexcept SttLoadCancelledError:\n    engine = None  # retry after training settles\n    schedule_stt_load_retry()","preventionTips":["Do not start STT loads while a training run is being admitted","Catch and distinguish SttLoadCancelledError from real load failures","Retry loads on idle (no queued training) to avoid preemption loops"],"tags":["stt","cancellation","training","gpu","preemption"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}