{"record":{"id":"2e855ccf632fba41","repo":"aaif-goose/goose","slug":"llama-cpp-model-is-missing-a-quantization","errorCode":null,"errorMessage":"llama.cpp model '{}' is missing a quantization","messagePattern":"llama\\.cpp model '(.+?)' is missing a quantization","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/goose-local-inference/src/hf_models.rs","lineNumber":1988,"sourceCode":"\nfn mlx_variant_label(variant_id: &str) -> String {\n    if variant_id == MLX_VARIANT_ID {\n        \"MLX\".to_string()\n    } else {\n        format!(\"MLX {}\", variant_id.to_uppercase())\n    }\n}\n\npub async fn resolve_local_model_selection(\n    repo_id: &str,\n    backend_id: &str,\n    variant_id: Option<&str>,\n) -> Result<ResolvedLocalModel> {\n    match backend_id {\n        MLX_BACKEND_ID => resolve_mlx_model(repo_id, variant_id.unwrap_or(MLX_VARIANT_ID)).await,\n        LLAMACPP_BACKEND_ID => {\n            let quantization = variant_id.ok_or_else(|| {\n                anyhow::anyhow!(\"llama.cpp model '{}' is missing a quantization\", repo_id)\n            })?;\n            resolve_gguf_model(repo_id, quantization).await\n        }\n        _ => bail!(\"Unknown local inference backend '{}'\", backend_id),\n    }\n}\n\nfn snapshot_root_for_file(\n    path: &std::path::Path,\n    repo_filename: &str,\n) -> Option<std::path::PathBuf> {\n    let mut root = path.to_path_buf();\n    for _ in 0..repo_filename.split('/').count() {\n        root.pop();\n    }\n    Some(root)\n}\n","sourceCodeStart":1970,"sourceCodeEnd":2006,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/crates/goose-local-inference/src/hf_models.rs#L1970-L2006","documentation":"resolve_local_model_selection dispatches on backend: MLX gets a default variant (unwrap_or(MLX_VARIANT_ID)) but the llama.cpp/GGUF path requires variant_id because the variant string is the quantization that selects the actual file. Passing None with LLAMACPP_BACKEND_ID bails here before resolve_gguf_model runs.","triggerScenarios":"Calling resolve_local_model_selection(repo, \"llamacpp\" backend id, None); typically a UI selection or persisted config that omitted the quantization, or a caller assuming a default quant exists.","commonSituations":"Stored model configs predating a schema that added the variant field; selection objects built from search results that lack a chosen quant; programmatic calls copying the MLX pattern where the variant is optional.","solutions":["Pass the quantization as variant_id, e.g. 'Q4_K_M', 'Q8_0', 'BF16'","Enumerate the repo's available quantizations (the GGUF file list) and have the caller choose one explicitly","Migrate stale configs to always persist the variant for llama.cpp models"],"exampleFix":"// before\nlet resolved = resolve_local_model_selection(repo, LLAMACPP_BACKEND_ID, None).await?;\n\n// after\nlet quant = variant_id.context(\"select a quantization for llama.cpp models\")?;\nlet resolved = resolve_local_model_selection(repo, LLAMACPP_BACKEND_ID, Some(quant)).await?;","handlingStrategy":"validation","validationCode":"// llama.cpp models need an explicit quantization variant\nif backend_id == LLAMACPP_BACKEND_ID {\n    anyhow::ensure!(variant_id.is_some_and(|v| !v.is_empty()),\n        \"pick a quantization (e.g. Q4_K_M) for llama.cpp model {repo_id}\");\n}","typeGuard":"fn selection_is_complete(backend_id: &str, variant_id: Option<&str>) -> bool {\n    if backend_id == LLAMACPP_BACKEND_ID { variant_id.is_some_and(|v| !v.is_empty()) } else { true }\n}","tryCatchPattern":"match resolve_local_model_selection(repo, backend, variant).await {\n    Err(e) if e.to_string().contains(\"missing a quantization\") => {\n        // open a quantization picker fed by the repo's GGUF list, then retry\n    }\n    other => other,\n}","preventionTips":["Persist the quantization alongside the repo id for every llama.cpp model","Do not mirror the MLX flow where the variant is optional","Validate selections at the UI boundary so backend resolvers never see None"],"tags":["rust","validation","gguf","llamacpp","local-inference"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}