{"record":{"id":"b0613670b09f19cb","repo":"aaif-goose/goose","slug":"llama-cpp-model-is-missing-a-quantization-b06136","errorCode":null,"errorMessage":"llama.cpp model '{}' is missing a quantization","messagePattern":"llama\\.cpp model '(.+?)' is missing a quantization","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/goose-local-inference/src/management.rs","lineNumber":707,"sourceCode":"                variant_id,\n            })),\n            _ => anyhow::bail!(\"Unknown local inference backend '{}'\", backend_id),\n        }\n    } else {\n        Ok(None)\n    }\n}\n\nasync fn local_model_id_from_request(\n    req: &LocalInferenceModelDownloadRequest,\n    selection: Option<&LocalModelSelection>,\n) -> Result<String> {\n    if let Some(selection) = selection {\n        return match selection.backend_id.as_str() {\n            \"mlx\" => Ok(selection.repo_id.clone()),\n            \"llamacpp\" => {\n                let quantization = selection.variant_id.as_deref().ok_or_else(|| {\n                    anyhow!(\n                        \"llama.cpp model '{}' is missing a quantization\",\n                        selection.repo_id\n                    )\n                })?;\n                Ok(model_id_from_repo(&selection.repo_id, quantization))\n            }\n            _ => anyhow::bail!(\"Unknown local inference backend '{}'\", selection.backend_id),\n        };\n    }\n\n    if let Ok((repo_id, quantization)) = hf_models::parse_model_spec(&req.spec) {\n        return Ok(model_id_from_repo(&repo_id, &quantization));\n    }\n\n    let variants = hf_models::get_repo_local_variants(&req.spec).await?;\n    let has_llamacpp = variants\n        .iter()\n        .any(|variant| variant.backend_id == \"llamacpp\");","sourceCodeStart":689,"sourceCodeEnd":725,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/crates/goose-local-inference/src/management.rs#L689-L725","documentation":"When a download request selects the llamacpp backend, local_model_id_from_request() requires a quantization: either selection.variant_id or a spec that parses as 'owner/repo:QUANT'. With neither present it bails with this error. The quantization is needed because llama.cpp model ids embed the quant (model_id_from_repo builds 'owner/repo:QUANT'). MLX selections don't need one.","triggerScenarios":"backend_id=\"llamacpp\" with spec=\"Qwen/Qwen2.5-7B\" (no :Q4_K_M suffix) and variant_id=None. Also when the spec's suffix is not parseable by parse_model_spec, so variant_id stays None.","commonSituations":"Users copying the bare HuggingFace repo name; UI sending the repo id in spec and forgetting the variant picker; prompting an LLM agent that fills spec without the quant suffix.","solutions":["Qualify the spec with a quantization: \"owner/repo:Q4_K_M\" (Q4_K_M, Q5_K_M, Q8_0, etc.).","Or pass variant_id explicitly (e.g. \"Q4_K_M\") alongside backend_id=\"llamacpp\".","Or drop backend_id and let auto-detection resolve the variant from the repo's GGUF files.","Choose \"mlx\" instead if you want to download by bare repo id."],"exampleFix":"// before\nlet req = DownloadRequest { spec: \"Qwen/Qwen2.5-7B\".into(), backend_id: Some(\"llamacpp\".into()), variant_id: None, ..Default::default() };\n// -> llama.cpp model 'Qwen/Qwen2.5-7B' is missing a quantization\n\n// after\nlet req = DownloadRequest { spec: \"Qwen/Qwen2.5-7B:Q4_K_M\".into(), backend_id: Some(\"llamacpp\".into()), variant_id: None, ..Default::default() };","handlingStrategy":"validation","validationCode":"if req.backend_id.as_deref() == Some(\"llamacpp\") && req.variant_id.is_none() {\n    if let Ok((_, quant)) = hf_models::parse_model_spec(&req.spec) {\n        req.variant_id = Some(quant);\n    } else {\n        anyhow::bail!(\"llamacpp downloads need a quantization, e.g. 'owner/repo:Q4_K_M'\");\n    }\n}\nlet id = resolve_model_id(&req).await?;","typeGuard":"fn has_llamacpp_quantization(spec: &str, variant_id: Option<&str>) -> bool {\n    variant_id.is_some() || hf_models::parse_model_spec(spec).is_ok()\n}","tryCatchPattern":null,"preventionTips":["Always send llama.cpp specs as 'owner/repo:QUANT' (e.g. ':Q4_K_M').","Set variant_id explicitly when spec and quant are provided separately.","Prefer dropping backend_id and letting auto-detection pick the quant."],"tags":["rust","validation","quantization","llamacpp","download","local-inference"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}