cjpais/Handy · error · anyhow::Error

Failed to load whisper model {}: {}

Error message

Failed to load whisper model {}: {}

What it means

transcribe-cpp's Model::load_with failed to load the Whisper-family GGUF with the selected backend and gpu_device (bound via select_transcribe_backend/resolve_gpu_device). The embedded inner error discriminates the cause: unsupported GGUF version or architecture, corrupted or truncated file, out-of-memory, or a GPU backend that cannot initialize. A loading_failed event is emitted with the composed message and the previously loaded engine has already been dropped.

Source

Thrown at src-tauri/src/managers/transcription.rs:575

                        emit_loading_failed(&e.to_string());
                    })?,
                    None => {
                        let settings = get_settings(&self.app_handle);
                        let accelerator = settings.transcribe_accelerator;
                        (
                            select_transcribe_backend(accelerator),
                            resolve_gpu_device(accelerator, settings.transcribe_gpu_device),
                        )
                    }
                };
                let model_options = ModelOptions {
                    backend,
                    gpu_device,
                };
                let model = Model::load_with(&model_path, &model_options).map_err(|e| {
                    let error_msg = format!("Failed to load whisper model {}: {}", model_id, e);
                    emit_loading_failed(&error_msg);
                    anyhow::anyhow!(error_msg)
                })?;
                // The bound backend may differ from the request (e.g. CPU
                // fallback under Auto); log what actually loaded.
                let bound_backend = model.backend();
                let session = model.session().map_err(|e| {
                    let error_msg = format!(
                        "Failed to create session for whisper model {}: {}",
                        model_id, e
                    );
                    emit_loading_failed(&error_msg);
                    anyhow::anyhow!(error_msg)
                })?;
                // Reconcile the registry's advertised capabilities with the
                // loaded model's real ones (GGUF metadata) so badges/gating
                // reflect runtime truth, not the pre-download probe. The
                // load-completed event below triggers the frontend refresh.
                let caps = session.model().capabilities();
                self.model_manager.set_runtime_capabilities(

View on GitHub (pinned to 98a4d80cce)

Solutions

1. Read the embedded inner error first — it separates unsupported-format from OOM from backend failure
2. Switch the accelerator setting to CPU and retry to rule out Vulkan/Metal/driver issues
3. Free memory (close other apps) or choose a smaller model size/quantization
4. Re-download the model (delete_model + download_model) to rule out corruption, and update Handy for newer transcribe-cpp GGUF support

Example fix

// before
let model = Model::load_with(&model_path, &model_options)?;

// after — retry the same file on CPU when a GPU backend fails to load
let model = match Model::load_with(&model_path, &model_options) {
    Ok(m) => m,
    Err(e) if !matches!(model_options.backend, Backend::Cpu) => {
        warn!("GPU load failed ({}), falling back to CPU", e);
        Model::load_with(&model_path, &ModelOptions { backend: Backend::Cpu, gpu_device: 0 })?
    }
    Err(e) => return Err(anyhow::anyhow!("Failed to load whisper model {}: {}", model_id, e)),
};

// Sanity-check the GGUF before loading: magic + non-trivial size
fn looks_like_valid_gguf(path: &Path) -> bool {
    let mut magic = [0u8; 4];
    let Ok(mut f) = std::fs::File::open(path) else { return false };
    use std::io::Read;
    f.read_exact(&mut magic).is_ok() && &magic == b"GGUF" && path.metadata().map(|m| m.len() > 1_000_000).unwrap_or(false)
}

let opts = ModelOptions { backend, gpu_device };
let model = match Model::load_with(&model_path, &opts) {
    Ok(m) => m,
    Err(e) if !matches!(backend, Backend::Cpu) => {
        // GPU/backend load failure: degrade to CPU rather than failing the feature
        emit_loading_failed(&format!("GPU load failed ({}), retrying on CPU", e));
        Model::load_with(&model_path, &ModelOptions { backend: Backend::Cpu, gpu_device: 0 })?
    }
    Err(e) => return Err(anyhow::anyhow!("Failed to load whisper model {}: {}", model_id, e)),
};

When it happens

Trigger: GGUF written by a newer converter than the vendored transcribe-cpp supports; a custom drop-in model with an unknown architecture; RAM/VRAM exhaustion on large models; Vulkan/Metal driver problems under the Auto accelerator; file locked by antivirus.

Common situations: Manually dropping in bleeding-edge GGUF quantizations; low-memory machines loading large or turbo models; broken GPU drivers after an OS update; loading right after a download that passed size but had no pinned hash.

Related errors

• Failed to create session for whisper model {}: {}
• transcribe-cpp transcription failed: {}
• Failed to load parakeet model {}: {}
• Failed to load moonshine model {}: {}
• Failed to load moonshine streaming model {}: {}

AI-assisted analysis of cjpais/Handy@98a4d80cce (2026-08-16). Data as JSON: /api/errors/15372a202bd23cb3. Report an issue: GitHub.