What causes "llama.cpp model '{}' is missing a quantization" in aaif-goose/goose?

Pass the quantization as variant_id, e.g. 'Q4_K_M', 'Q8_0', 'BF16'

llama.cpp model '{}' is missing a quantization

Error message

llama.cpp model '{}' is missing a quantization

What it means

resolve_local_model_selection dispatches on backend: MLX gets a default variant (unwrap_or(MLX_VARIANT_ID)) but the llama.cpp/GGUF path requires variant_id because the variant string is the quantization that selects the actual file. Passing None with LLAMACPP_BACKEND_ID bails here before resolve_gguf_model runs.

Source

Thrown at crates/goose-local-inference/src/hf_models.rs:1988

fn mlx_variant_label(variant_id: &str) -> String {
    if variant_id == MLX_VARIANT_ID {
        "MLX".to_string()
    } else {
        format!("MLX {}", variant_id.to_uppercase())
    }
}

pub async fn resolve_local_model_selection(
    repo_id: &str,
    backend_id: &str,
    variant_id: Option<&str>,
) -> Result<ResolvedLocalModel> {
    match backend_id {
        MLX_BACKEND_ID => resolve_mlx_model(repo_id, variant_id.unwrap_or(MLX_VARIANT_ID)).await,
        LLAMACPP_BACKEND_ID => {
            let quantization = variant_id.ok_or_else(|| {
                anyhow::anyhow!("llama.cpp model '{}' is missing a quantization", repo_id)
            })?;
            resolve_gguf_model(repo_id, quantization).await
        }
        _ => bail!("Unknown local inference backend '{}'", backend_id),
    }
}

fn snapshot_root_for_file(
    path: &std::path::Path,
    repo_filename: &str,
) -> Option<std::path::PathBuf> {
    let mut root = path.to_path_buf();
    for _ in 0..repo_filename.split('/').count() {
        root.pop();
    }
    Some(root)
}

View on GitHub (pinned to 3810898a74)

Solutions

Pass the quantization as variant_id, e.g. 'Q4_K_M', 'Q8_0', 'BF16'
Enumerate the repo's available quantizations (the GGUF file list) and have the caller choose one explicitly
Migrate stale configs to always persist the variant for llama.cpp models

Example fix

// before
let resolved = resolve_local_model_selection(repo, LLAMACPP_BACKEND_ID, None).await?;

// after
let quant = variant_id.context("select a quantization for llama.cpp models")?;
let resolved = resolve_local_model_selection(repo, LLAMACPP_BACKEND_ID, Some(quant)).await?;

Defensive patterns

Strategy: validation

Validate before calling

// llama.cpp models need an explicit quantization variant
if backend_id == LLAMACPP_BACKEND_ID {
    anyhow::ensure!(variant_id.is_some_and(|v| !v.is_empty()),
        "pick a quantization (e.g. Q4_K_M) for llama.cpp model {repo_id}");
}

Type guard

fn selection_is_complete(backend_id: &str, variant_id: Option<&str>) -> bool {
    if backend_id == LLAMACPP_BACKEND_ID { variant_id.is_some_and(|v| !v.is_empty()) } else { true }
}

Try / catch

match resolve_local_model_selection(repo, backend, variant).await {
    Err(e) if e.to_string().contains("missing a quantization") => {
        // open a quantization picker fed by the repo's GGUF list, then retry
    }
    other => other,
}

Prevention

Persist the quantization alongside the repo id for every llama.cpp model
Do not mirror the MLX flow where the variant is optional
Validate selections at the UI boundary so backend resolvers never see None

When it happens

Trigger: Calling resolve_local_model_selection(repo, "llamacpp" backend id, None); typically a UI selection or persisted config that omitted the quantization, or a caller assuming a default quant exists.

Common situations: Stored model configs predating a schema that added the variant field; selection objects built from search results that lack a chosen quant; programmatic calls copying the MLX pattern where the variant is optional.

Related errors

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/2e855ccf632fba41. Report an issue: GitHub.