cjpais/Handy · error · anyhow::Error

Failed to create session for whisper model {}: {}

Error message

Failed to create session for whisper model {}: {}

What it means

The GGUF file loaded, but creating the inference session (model.session()) failed — the backend could not initialize its compute context. Causes sit mostly at the GPU layer: Vulkan/Metal device or context creation failure, VRAM exhaustion at context init, or driver instability. The error is distinct from load failure: the file itself was parsed successfully, and runtime capabilities reconciliation never runs because the session is unavailable.

Source

Thrown at src-tauri/src/managers/transcription.rs:586

                let model_options = ModelOptions {
                    backend,
                    gpu_device,
                };
                let model = Model::load_with(&model_path, &model_options).map_err(|e| {
                    let error_msg = format!("Failed to load whisper model {}: {}", model_id, e);
                    emit_loading_failed(&error_msg);
                    anyhow::anyhow!(error_msg)
                })?;
                // The bound backend may differ from the request (e.g. CPU
                // fallback under Auto); log what actually loaded.
                let bound_backend = model.backend();
                let session = model.session().map_err(|e| {
                    let error_msg = format!(
                        "Failed to create session for whisper model {}: {}",
                        model_id, e
                    );
                    emit_loading_failed(&error_msg);
                    anyhow::anyhow!(error_msg)
                })?;
                // Reconcile the registry's advertised capabilities with the
                // loaded model's real ones (GGUF metadata) so badges/gating
                // reflect runtime truth, not the pre-download probe. The
                // load-completed event below triggers the frontend refresh.
                let caps = session.model().capabilities();
                self.model_manager.set_runtime_capabilities(
                    model_id,
                    caps.supports_streaming,
                    caps.supports_translate,
                    caps.supports_language_detect,
                    caps.languages.clone(),
                );
                info!(
                    "Loaded whisper model '{}' (requested {:?}, gpu_device {}, bound backend '{}', \
                     supports_streaming={}, supports_translate={}, supports_language_detect={})",
                    model_id,
                    backend,

View on GitHub (pinned to 98a4d80cce)

Solutions

1. Switch the accelerator to CPU, or select a different gpu_device index, and retry
2. Ensure the Vulkan runtime is present and healthy (vulkaninfo) and update GPU drivers
3. Free VRAM: close other GPU-heavy apps or use a smaller model
4. Check logs for the embedded backend error and report it if it persists on healthy drivers

// Cheap device sanity check before selecting a GPU backend
fn gpu_backend_likely_ok(accelerator: &Accelerator) -> bool {
    match accelerator {
        Accelerator::Cpu => true,
        Accelerator::Vulkan => vulkan_available(), // e.g. vulkaninfo succeeds
        Accelerator::Auto => true,                 // auto can fall back internally
    }
}

let session = match model.session() {
    Ok(s) => s,
    Err(e) => {
        // context init failed on the bound GPU backend: rebuild model on CPU
        drop(model);
        let cpu_opts = ModelOptions { backend: Backend::Cpu, gpu_device: 0 };
        let cpu_model = Model::load_with(&model_path, &cpu_opts)?;
        cpu_model.session().map_err(|e2| anyhow::anyhow!(
            "session failed on GPU ({}) and CPU ({})", e, e2))?  
    }
};

When it happens

Trigger: Vulkan context creation failure (missing or broken drivers, headless session without a GPU); VRAM exhausted by another application; device lost during initialization; CPU backend with insufficient RAM for context buffers.

Common situations: Linux systems with incomplete Vulkan stacks; GPUs near VRAM capacity; driver updates mid-session; multi-GPU systems where the selected transcribe_gpu_device index is unavailable.

Related errors

• transcribe-cpp transcription failed: {}
• Failed to load whisper model {}: {}
• No compute device with index {index} (see --list-devices)
• Device index {index} has kind '{other}', which cannot host a

AI-assisted analysis of cjpais/Handy@98a4d80cce (2026-08-16). Data as JSON: /api/errors/5f6a5e53fa14e07e. Report an issue: GitHub.