unslothai/unsloth · info · SttLoadCancelledError

STT model loading was cancelled so training could start.

Error message

STT model loading was cancelled so training could start.

What it means

Raised in the STT worker child process by _raise_if_cancelled when the shared cancel event is set during model loading. The parent sets the event to abort a long Whisper load so GPU/memory can be handed to training. SttLoadCancelledError signals deliberate preemption, not corruption or a broken model.

Source

Thrown at studio/backend/core/inference/stt_transformers_worker.py:97

def _ensure_backend_on_path() -> None:
    if _BACKEND_PATH not in sys.path:
        sys.path.insert(0, _BACKEND_PATH)


class _CancelCriteria:
    """Stops generation as soon as the parent sets the shared cancel event."""

    def __init__(self, cancel_event) -> None:
        self._cancel_event = cancel_event

    def __call__(self, *_args, **_kwargs) -> bool:
        return self._cancel_event.is_set()


def _raise_if_cancelled(cancel_event) -> None:
    from core.inference.stt_sidecar import SttLoadCancelledError
    if cancel_event is not None and cancel_event.is_set():
        raise SttLoadCancelledError("STT model loading was cancelled so training could start.")


def load_whisper(
    snapshot_path: str,
    device: str,
    dtype_name: str,
    cancel_event = None,
) -> tuple:
    """Load a Whisper model + processor from the local Hub cache. Child side.

    local_files_only keeps the Model Hub the only download path; a cache miss
    raises so the parent can surface SttModelNotDownloadedError.
    """
    import torch
    from transformers import WhisperForConditionalGeneration, WhisperProcessor

    dtype = getattr(torch, dtype_name, None) or torch.float32
    processor = WhisperProcessor.from_pretrained(snapshot_path, local_files_only = True)

View on GitHub (pinned to 203007d190)

Solutions

1. Catch SttLoadCancelledError in the parent and retry the load when the GPU is free (optionally via the existing load API with a fresh event).
2. If loads are cancelled repeatedly, schedule STT loads outside training windows or on a quieter device.
3. Do not reuse the cancelled event; create a new one for the retry.

Example fix

# before
worker.load(model_id, cancel_event=evt)  # raises mid-load when training starts

# after
try:
    worker.load(model_id, cancel_event=evt)
except SttLoadCancelledError:
    evt = threading.Event()  # fresh event
    worker.load(model_id, cancel_event=evt)  # retry after training frees the GPU

from core.inference.stt_sidecar import SttLoadCancelledError
try:
    stt.load(model_id, cancel_event=evt)
except SttLoadCancelledError:
    evt = threading.Event()
    schedule_retry_when_gpu_free(lambda: stt.load(model_id, cancel_event=evt))

When it happens

Trigger: A load_whisper(...) call in the worker child with a cancel_event that the parent process sets while weights are still being read from the Hub cache and moved to the device.

Common situations: A scheduler that preempts STT model loads when a training job starts; low-VRAM hosts where loads are slow and often interrupted; rapid switching between dictation and training features.

Related errors

• STT model loading was cancelled so training could start.
• '{model_id}' is still cancelling; try again in a moment.
• Transcription cancelled.
• GGUF STT model loading was cancelled.
• GGUF STT model loading was cancelled so training could start

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/38efab7d2c64d6b3. Report an issue: GitHub.