cjpais/Handy · error · anyhow::Error

Device index {index} ({}) cannot host a model

Error message

Device index {index} ({}) cannot host a model

What it means

Handy throws this from resolve_device_index (src-tauri/src/managers/transcription.rs:1940) when an explicit device selection (--device-index N, or load_model_with_device(model_id, Some(n))) resolves to a registered transcribe-cpp device whose DeviceType is Accel or Unknown. The index itself is valid — it matched an entry in transcribe_compute_devices() (otherwise the 'No compute device with index' error at line 1934 fires) — but the matched entry is a helper accelerator or an unclassifiable device, and only primary compute devices (CPU, Gpu, Igpu) can host a Whisper-family GGML model. The model load is aborted: a 'loading_failed' model-state event is emitted and the headless --transcribe-file run exits with the error.

Source

Thrown at src-tauri/src/managers/transcription.rs:1940

        .collect()
}

/// Resolve a `--list-devices` registry index to an exact opaque device handle
/// for a transcribe-cpp model load (the `--device-index` flag). In 0.2 index 0
/// is an exact selection too; only an omitted index requests automatic device
/// selection. Errors if the index isn't a registered, loadable primary device.
fn resolve_device_index(index: usize) -> Result<(Backend, Option<transcribe_cpp::Device>)> {
    let device = transcribe_compute_devices()
        .into_iter()
        .find(|d| d.index == Some(index))
        .ok_or_else(|| {
            anyhow::anyhow!("No compute device with index {index} (see --list-devices)")
        })?;
    if matches!(
        device.device_type,
        transcribe_cpp::DeviceType::Accel | transcribe_cpp::DeviceType::Unknown
    ) {
        return Err(anyhow::anyhow!(
            "Device index {index} ({}) cannot host a model",
            device.kind
        ));
    }

    // 0.2's opaque handle makes every index, including zero, an exact
    // selection. Backend::Auto accepts any primary device and cannot conflict
    // with the selected device's vendor backend.
    Ok((Backend::Auto, Some(device)))
}

/// Map Handy's whisper accelerator setting to a transcribe-cpp [`Backend`].
///
/// `Auto` lets the library pick the best device (with CPU fallback), while
/// `Cpu` forces strict CPU. `Gpu` only remains as the companion setting for an
/// exact device; without a valid exact device it has the retired generic GPU
/// state's new Auto semantics. An emulated x64 process on Windows ARM64 forces
/// strict CPU for every setting.

View on GitHub (pinned to fbd4e15fa1)

Solutions

1. Run `handy --list-devices` and re-check the index you passed: pick an entry whose kind is cpu, gpu, or igpu (vram>0MB usually indicates a real GPU), never kind=accel or an entry with no recognizable kind.
2. Omit --device-index entirely and let the persisted accelerator setting pick (Backend::Auto handles CPU fallback); you only need an explicit index for exact multi-GPU selection.
3. If you expected a GPU at that index, check the startup log line 'transcribe-cpp initialized with N compute device(s): [...]' — a 'Failed to initialize transcribe-cpp backends' warning means fewer devices registered, shifting indices; fix the backend init (drivers, Vulkan/Metal availability) and re-list devices.
4. On Windows x64 running emulated on an ARM64 host, GPU devices are intentionally filtered out (only cpu/accel remain), so no index will select a GPU — run a native ARM64 build or omit --device-index to use CPU.
5. For API callers of load_model_with_device: validate the index against transcribe_cpp::devices() and reject DeviceType::Accel/Unknown before invoking the load (see validation code below).

Example fix

# before — index 1 is an accel entry, load aborts
$ handy --list-devices
index=0 kind=cpu name=AMD Ryzen ... vram=0MB
index=1 kind=accel name=... vram=0MB
index=2 kind=gpu name=NVIDIA GeForce RTX ... vram=8192MB
$ handy --transcribe-file audio.wav --device-index 1
Error: Device index 1 (accel) cannot host a model

# after — select a primary device (gpu/cpu), or drop the flag for automatic selection
$ handy --transcribe-file audio.wav --device-index 2
$ handy --transcribe-file audio.wav   # uses persisted accelerator setting (Auto)

// Run before load_model_with_device(model_id, Some(index)) — mirrors
// resolve_device_index's rule: index must exist AND be a primary device.
use transcribe_cpp::DeviceType;

fn device_index_can_host_model(index: usize) -> bool {
    transcribe_cpp::devices()
        .into_iter()
        .any(|d| d.index == Some(index)
            && !matches!(d.device_type, DeviceType::Accel | DeviceType::Unknown))
}

// before loading:
if !device_index_can_host_model(idx) {
    // re-enumerate, log describe_compute_devices(), and fall back to None
    // (automatic selection) or fail fast with a clear message
}
tm.load_model_with_device(&model_id, Some(idx))?;

// Rust predicate narrowing a Device to 'can host a model' (primary compute).
fn is_primary_compute_device(d: &transcribe_cpp::Device) -> bool {
    !matches!(
        d.device_type,
        transcribe_cpp::DeviceType::Accel | transcribe_cpp::DeviceType::Unknown
    )
}

// If you still call with an explicit index, catch and retry once with
// automatic selection so a shifted registry doesn't kill the job:
match tm.load_model_with_device(&model_id, Some(idx)) {
    Ok(()) => {}
    Err(e) if e.to_string().contains("cannot host a model") => {
        log::warn!("device index {idx} unusable, retrying with automatic selection: {e}");
        tm.load_model_with_device(&model_id, None)?;
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Running `handy --transcribe-file audio.wav --device-index N` where N is the --list-devices index of an entry with kind=accel or an unknown-type entry; or calling TranscriptionManager::load_model_with_device(model_id, Some(n)) in Rust with such an index. Concretely: the device registry lists something like `index=1 kind=accel name=... vram=0MB` and you pass --device-index 1. Also triggered when the registry shrank or shifted between runs — e.g. on a Windows x64 build running emulated on an ARM64 host, GPU entries are filtered out of transcribe_compute_devices() (transcription.rs:2076-2087) leaving only cpu/accel kinds, so an index that used to hit a GPU now hits an accel entry; or a backend init failure (logged as 'Failed to initialize transcribe-cpp backends') changed which devices registered.

Common situations: Hard-coding a device index in a script/WM config that was observed on a different machine or an earlier run — indices are process-local and shift when drivers, eGPU hot-plug, or backend availability change. Machines that register accelerator entries (kind=accel) alongside GPU/CPU, where the user picks the wrong line from --list-devices output. Windows-on-ARM emulation setups where the GPU entries are hidden and users assume index 1 is still the discrete GPU. Storing a raw registry index in automation instead of a stable device identity (the settings layer deliberately stores a device key, not an index).

Related errors

• Failed to load whisper model {}: {}
• Failed to create session for whisper model {}: {}
• transcribe-cpp transcription failed: {}
• get_available_accelerators panicked
• No compute device with index {index} (see --list-devices)

AI-assisted analysis of cjpais/Handy@fbd4e15fa1 (2026-08-22). Data as JSON: /api/errors/4553066d11093870. Report an issue: GitHub.