{"record":{"id":"8514524bf257c7d3","repo":"huggingface/candle","slug":"softmax-last-dim-is-not-implemented-for-dtype","errorCode":null,"errorMessage":"softmax-last-dim is not implemented for {dtype:?}","messagePattern":"softmax-last-dim is not implemented for (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":404,"sourceCode":"        Ok((dst, layout.shape().clone()))\n    }\n\n    #[cfg(feature = \"metal\")]\n    fn metal_fwd(\n        &self,\n        storage: &candle::MetalStorage,\n        layout: &Layout,\n    ) -> Result<(candle::MetalStorage, Shape)> {\n        use candle::backend::BackendStorage;\n        let device = storage.device();\n        let encoder = device.command_encoder()?;\n        encoder.set_label(\"softmax\");\n        let kernels = device.kernels();\n        let name = match storage.dtype() {\n            DType::F32 => \"softmax_f32\",\n            DType::F16 => \"softmax_f16\",\n            DType::BF16 => \"softmax_bf16\",\n            dtype => candle::bail!(\"softmax-last-dim is not implemented for {dtype:?}\"),\n        };\n\n        let n = layout.stride().len();\n        if !(layout.is_contiguous() && layout.stride()[n - 1] == 1) {\n            candle::bail!(\"Non contiguous softmax-last-dim is not implemented\");\n        }\n\n        let last_dim = layout.dims()[layout.shape().rank() - 1];\n        let elem_count = layout.shape().elem_count();\n        let output = device\n            .new_buffer_builder()\n            .with_size_for(elem_count, storage.dtype())\n            .with_label(\"softmax\")\n            .build()?;\n        candle_metal_kernels::call_last_softmax(\n            device.metal_device(),\n            &encoder,\n            kernels,","sourceCodeStart":386,"sourceCodeEnd":422,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L386-L422","documentation":"The Metal softmax kernel dispatches to named metal shaders softmax_f32/softmax_f16/softmax_bf16 only. Any other dtype (F64, integers, etc.) has no GPU kernel registered, so metal_fwd bails with this message naming the dtype.","triggerScenarios":"Calling ops::softmax_last_dim on a Metal-device tensor with a dtype other than F32/F16/BF16, such as F64 or I64.","commonSituations":"Default F64 tensors created from Rust f64 arrays and sent to a Metal device; index/integer tensors accidentally passed to softmax; dtype changed upstream by a model config.","solutions":["Cast to F32 (or F16/BF16) with .to_dtype() before the softmax","Keep model tensors in F32/F16/BF16 on Metal; avoid F64 on GPU backends","Verify the op's device matches the intended backend; CPU supports F64 softmax but Metal does not"],"exampleFix":"// before\nlet probs = softmax_last_dim(&logits.to_device(&metal_device)?)?; // F64\n// after\nlet probs = softmax_last_dim(&logits.to_dtype(DType::F32)?.to_device(&metal_device)?)?;","handlingStrategy":"validation","validationCode":"fn ensure_metal_softmax(t: &Tensor) -> Result<()> {\n    if !matches!(t.dtype(), DType::F32 | DType::F16 | DType::BF16) {\n        bail!(\"Metal softmax supports F32/F16/BF16 only, got {:?}\", t.dtype());\n    }\n    Ok(())\n}","typeGuard":"fn metal_softmax_ok(d: DType) -> bool { matches!(d, DType::F32 | DType::F16 | DType::BF16) }","tryCatchPattern":"let t = if metal_softmax_ok(t.dtype()) { t } else { t.to_dtype(DType::F32)? };","preventionTips":["Never use F64 tensors on Metal devices","Keep GPU pipeline dtypes uniform (F32 or F16/BF16)","Cast at device-transfer time: to_dtype + to_device together"],"tags":["metal","softmax","dtype","gpu","candle"],"backgroundTag":"unsupported-dtype-for-op","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}