{"record":{"id":"c79323dced8ac9ff","repo":"aaif-goose/goose","slug":"local-inference-with-the-bundled-llama-cpp-backend","errorCode":null,"errorMessage":"Local inference with the bundled llama.cpp backend requires CPU support for {}. This CPU is missing {}. Use a CPU with those instruction sets or switch to a non-local provider.","messagePattern":"Local inference with the bundled llama\\.cpp backend requires CPU support for (.+?)\\. This CPU is missing (.+?)\\. Use a CPU with those instruction sets or switch to a non-local provider\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/goose-local-inference/src/llamacpp/mod.rs","lineNumber":332,"sourceCode":"}\n\n#[cfg(target_arch = \"x86_64\")]\nfn check_cpu_supports_local_inference() -> Result<()> {\n    let missing_features = [\n        (!std::arch::is_x86_feature_detected!(\"fma\")).then_some(\"FMA\"),\n        (!std::arch::is_x86_feature_detected!(\"avx2\")).then_some(\"AVX2\"),\n        (!std::arch::is_x86_feature_detected!(\"f16c\")).then_some(\"F16C\"),\n        (!std::arch::is_x86_feature_detected!(\"bmi2\")).then_some(\"BMI2\"),\n        (!std::arch::is_x86_feature_detected!(\"sse4.2\")).then_some(\"SSE4.2\"),\n    ]\n    .into_iter()\n    .flatten()\n    .collect::<Vec<_>>();\n\n    if missing_features.is_empty() {\n        Ok(())\n    } else {\n        Err(anyhow::anyhow!(unsupported_cpu_features_error_message(\n            &missing_features\n        )))\n    }\n}\n\n#[cfg(not(target_arch = \"x86_64\"))]\nfn check_cpu_supports_local_inference() -> Result<()> {\n    Ok(())\n}\n\npub(super) struct LlamaCppBackend {\n    backend: LlamaBackend,\n}\n\nimpl LlamaCppBackend {\n    pub(super) fn new() -> Result<Self> {\n        check_cpu_supports_local_inference()?;\n","sourceCodeStart":314,"sourceCodeEnd":350,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/crates/goose-local-inference/src/llamacpp/mod.rs#L314-L350","documentation":"On x86_64, LlamaCppBackend::new() first runs check_cpu_supports_local_inference, which probes FMA, AVX2, F16C, BMI2 and SSE4.2 with runtime feature detection and lists every missing set in the error. The bundled llama.cpp build is compiled against those instruction sets, so local llama.cpp inference refuses to start without them. Non-x86_64 targets skip the check entirely.","triggerScenarios":"Starting local inference (any llama.cpp/GGUF model) on a pre-~2013 Intel CPU, an old AMD/Atom chip, or a VM/container whose CPU feature mask hides AVX2 and friends. The error names the exact missing sets.","commonSituations":"Older home servers or office PCs; cloud VMs on older host generations (some cheap VPS lines lack AVX2); virtualization defaults that do not pass through extended flags; containers on such hosts.","solutions":["Verify with lscpu | grep -o 'avx2\\|fma\\|f16c\\|bmi2\\|sse4_2' which sets are present","Move to a host/VM that exposes all five feature sets (enable the CPU flags in hypervisor settings)","Otherwise switch the agent to a non-local (cloud) provider, as the message suggests"],"exampleFix":"// before: offer local inference unconditionally\nlet backend = LlamaCppBackend::new()?;\n\n// after: gate the local option on CPU capability first (see validation snippet), and surface a friendly provider-choice screen instead of the raw error","handlingStrategy":"validation","validationCode":"fn cpu_supports_llamacpp() -> bool {\n    #[cfg(target_arch = \"x86_64\")]\n    {\n        std::arch::is_x86_feature_detected!(\"fma\")\n            && std::arch::is_x86_feature_detected!(\"avx2\")\n            && std::arch::is_x86_feature_detected!(\"f16c\")\n            && std::arch::is_x86_feature_detected!(\"bmi2\")\n            && std::arch::is_x86_feature_detected!(\"sse4.2\")\n    }\n    #[cfg(not(target_arch = \"x86_64\"))]\n    { true }\n}\n// gate the local-inference option on this before the user selects a model","typeGuard":"fn localInferenceSupportedOnThisCpu() -> boolean {\n  // x86_64 only: all of fma, avx2, f16c, bmi2, sse4_2 must be present\n  return detectCpuFlags().every(f => ['fma','avx2','f16c','bmi2','sse4_2'].includes(f));\n}","tryCatchPattern":"match LlamaCppBackend::new() {\n    Err(e) if e.to_string().contains(\"requires CPU support for\") => {\n        // hide the local provider and guide the user to a cloud provider\n    }\n    other => other,\n}","preventionTips":["Check CPU flags (lscpu) before deploying local inference to a fleet","Enable full CPU feature passthrough in VM/container settings","Offer a provider fallback in the UI so unsupported CPUs never dead-end the user"],"tags":["rust","cpu","simd","llamacpp","local-inference","hardware"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}