{"record":{"id":"3331db8a0de97d62","repo":"huggingface/candle","slug":"sdpa-has-no-cpu-impl","errorCode":null,"errorMessage":"SDPA has no cpu impl","messagePattern":"SDPA has no cpu impl","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":1029,"sourceCode":"    mask: Option<Tensor>,\n    do_causal: bool,\n}\n\nimpl candle::CustomOp3 for Sdpa {\n    fn name(&self) -> &'static str {\n        \"metal-sdpa\"\n    }\n\n    fn cpu_fwd(\n        &self,\n        _s1: &CpuStorage,\n        _l1: &Layout,\n        _s2: &CpuStorage,\n        _l2: &Layout,\n        _s3: &CpuStorage,\n        _l3: &Layout,\n    ) -> Result<(CpuStorage, Shape)> {\n        candle::bail!(\"SDPA has no cpu impl\")\n    }\n\n    #[cfg(feature = \"metal\")]\n    fn metal_fwd(\n        &self,\n        q: &candle::MetalStorage,\n        q_l: &Layout,\n        k: &candle::MetalStorage,\n        k_l: &Layout,\n        v: &candle::MetalStorage,\n        v_l: &Layout,\n    ) -> Result<(candle::MetalStorage, Shape)> {\n        use candle::backend::BackendStorage;\n        use candle_metal_kernels::SdpaDType;\n\n        let device = q.device();\n\n        let out_dims = vec![q_l.dim(0)?, q_l.dim(1)?, q_l.dim(2)?, v_l.dim(3)?];","sourceCodeStart":1011,"sourceCodeEnd":1047,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L1011-L1047","documentation":"The Sdpa custom op in candle-nn is implemented only for the Metal backend; its cpu_fwd unconditionally bails. Scaled-dot-product attention via this fused op therefore cannot run when tensors are on the CPU device.","triggerScenarios":"Invoking the fused SDPA op (e.g. via sdpa helper in candle-nn) with CPU tensors, or running a model on CPU that hits the Sdpa CustomOp3 path.","commonSituations":"Running tests or inference on a Linux/Windows machine or explicitly with Device::Cpu; CI environments without Apple Silicon; debugging Metal code with a CPU device.","solutions":["Run on a Metal device (Device::new_metal) so metal_fwd is used","Replace the fused SDPA call with a manual attention implementation using standard Tensor ops (matmul, softmax) that work on CPU","Use a different backend build of candle (cuda/cudarc) if available, with its own SDPA implementation","Guard code paths so the fused SDPA is only reached when the device is Metal"],"exampleFix":"// before\nlet out = sdpa(&q, &k, &v, &mask, false, Some(1.0))?; // on CPU device\n// after\nlet dev = candle::Device::new_metal(0)?;\nlet q = q.to_device(&dev)?; // k, v, mask likewise\nlet out = sdpa(&q, &k, &v, &mask, false, Some(1.0))?;","handlingStrategy":"fallback","validationCode":"fn sdpa_device_ok(dev: &candle::Device) -> bool {\n    matches!(dev, candle::Device::Metal(_))\n}","typeGuard":"fn is_metal(dev: &candle::Device) -> bool { matches!(dev, candle::Device::Metal(_)) }","tryCatchPattern":"match try_fused_sdpa(&q, &k, &v) {\n    Ok(y) => y,\n    Err(e) if e.to_string().contains(\"no cpu impl\") => manual_attention(&q, &k, &v, scale)?,\n    Err(e) => return Err(e),\n}","preventionTips":["Only invoke fused SDPA on Metal devices","Keep a CPU-compatible manual attention path in the codebase","In CI (no Apple Silicon), test the manual attention path","Route model inference through a device check at startup"],"tags":["cpu","sdpa","unimplemented","candle"],"backgroundTag":"backend-not-implemented","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}