aaif-goose/goose · error

Local inference with the bundled llama.cpp backend requires

Error message

Local inference with the bundled llama.cpp backend requires CPU support for {}. This CPU is missing {}. Use a CPU with those instruction sets or switch to a non-local provider.

What it means

On x86_64, LlamaCppBackend::new() first runs check_cpu_supports_local_inference, which probes FMA, AVX2, F16C, BMI2 and SSE4.2 with runtime feature detection and lists every missing set in the error. The bundled llama.cpp build is compiled against those instruction sets, so local llama.cpp inference refuses to start without them. Non-x86_64 targets skip the check entirely.

Source

Thrown at crates/goose-local-inference/src/llamacpp/mod.rs:332

}

#[cfg(target_arch = "x86_64")]
fn check_cpu_supports_local_inference() -> Result<()> {
    let missing_features = [
        (!std::arch::is_x86_feature_detected!("fma")).then_some("FMA"),
        (!std::arch::is_x86_feature_detected!("avx2")).then_some("AVX2"),
        (!std::arch::is_x86_feature_detected!("f16c")).then_some("F16C"),
        (!std::arch::is_x86_feature_detected!("bmi2")).then_some("BMI2"),
        (!std::arch::is_x86_feature_detected!("sse4.2")).then_some("SSE4.2"),
    ]
    .into_iter()
    .flatten()
    .collect::<Vec<_>>();

    if missing_features.is_empty() {
        Ok(())
    } else {
        Err(anyhow::anyhow!(unsupported_cpu_features_error_message(
            &missing_features
        )))
    }
}

#[cfg(not(target_arch = "x86_64"))]
fn check_cpu_supports_local_inference() -> Result<()> {
    Ok(())
}

pub(super) struct LlamaCppBackend {
    backend: LlamaBackend,
}

impl LlamaCppBackend {
    pub(super) fn new() -> Result<Self> {
        check_cpu_supports_local_inference()?;

View on GitHub (pinned to 3810898a74)

Solutions

1. Verify with lscpu | grep -o 'avx2\|fma\|f16c\|bmi2\|sse4_2' which sets are present
2. Move to a host/VM that exposes all five feature sets (enable the CPU flags in hypervisor settings)
3. Otherwise switch the agent to a non-local (cloud) provider, as the message suggests

Example fix

// before: offer local inference unconditionally
let backend = LlamaCppBackend::new()?;

// after: gate the local option on CPU capability first (see validation snippet), and surface a friendly provider-choice screen instead of the raw error

fn cpu_supports_llamacpp() -> bool {
    #[cfg(target_arch = "x86_64")]
    {
        std::arch::is_x86_feature_detected!("fma")
            && std::arch::is_x86_feature_detected!("avx2")
            && std::arch::is_x86_feature_detected!("f16c")
            && std::arch::is_x86_feature_detected!("bmi2")
            && std::arch::is_x86_feature_detected!("sse4.2")
    }
    #[cfg(not(target_arch = "x86_64"))]
    { true }
}
// gate the local-inference option on this before the user selects a model

fn localInferenceSupportedOnThisCpu() -> boolean {
  // x86_64 only: all of fma, avx2, f16c, bmi2, sse4_2 must be present
  return detectCpuFlags().every(f => ['fma','avx2','f16c','bmi2','sse4_2'].includes(f));
}

match LlamaCppBackend::new() {
    Err(e) if e.to_string().contains("requires CPU support for") => {
        // hide the local provider and guide the user to a cloud provider
    }
    other => other,
}

When it happens

Trigger: Starting local inference (any llama.cpp/GGUF model) on a pre-~2013 Intel CPU, an old AMD/Atom chip, or a VM/container whose CPU feature mask hides AVX2 and friends. The error names the exact missing sets.

Common situations: Older home servers or office PCs; cloud VMs on older host generations (some cheap VPS lines lack AVX2); virtualization defaults that do not pass through extended flags; containers on such hosts.

Related errors

• llama.cpp model '{}' is missing a quantization
• Failed to init llama backend: {}
• Unknown local inference backend '{}'
• llama.cpp model '{}' is missing a quantization
• MLX model {} has no downloadable files

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/c79323dced8ac9ff. Report an issue: GitHub.