{"record":{"id":"017a7615ae46d414","repo":"huggingface/candle","slug":"non-contiguous-softmax-last-dim-is-not-implemented","errorCode":null,"errorMessage":"Non contiguous softmax-last-dim is not implemented","messagePattern":"Non contiguous softmax-last-dim is not implemented","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-nn/src/ops.rs","lineNumber":409,"sourceCode":"        &self,\n        storage: &candle::MetalStorage,\n        layout: &Layout,\n    ) -> Result<(candle::MetalStorage, Shape)> {\n        use candle::backend::BackendStorage;\n        let device = storage.device();\n        let encoder = device.command_encoder()?;\n        encoder.set_label(\"softmax\");\n        let kernels = device.kernels();\n        let name = match storage.dtype() {\n            DType::F32 => \"softmax_f32\",\n            DType::F16 => \"softmax_f16\",\n            DType::BF16 => \"softmax_bf16\",\n            dtype => candle::bail!(\"softmax-last-dim is not implemented for {dtype:?}\"),\n        };\n\n        let n = layout.stride().len();\n        if !(layout.is_contiguous() && layout.stride()[n - 1] == 1) {\n            candle::bail!(\"Non contiguous softmax-last-dim is not implemented\");\n        }\n\n        let last_dim = layout.dims()[layout.shape().rank() - 1];\n        let elem_count = layout.shape().elem_count();\n        let output = device\n            .new_buffer_builder()\n            .with_size_for(elem_count, storage.dtype())\n            .with_label(\"softmax\")\n            .build()?;\n        candle_metal_kernels::call_last_softmax(\n            device.metal_device(),\n            &encoder,\n            kernels,\n            name,\n            elem_count,\n            last_dim,\n            storage.buffer(),\n            layout.start_offset() * storage.dtype().size_in_bytes(),","sourceCodeStart":391,"sourceCodeEnd":427,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-nn/src/ops.rs#L391-L427","documentation":"The Metal softmax kernel requires the tensor to be contiguous with the last dimension having stride 1. Non-contiguous views (from transpose/permute/narrow) are not supported by the shader, so metal_fwd bails explicitly rather than producing wrong results.","triggerScenarios":"Calling ops::softmax_last_dim on a Metal tensor whose layout fails layout.is_contiguous() && stride()[last]==1 — typically after transpose/permute/slice.","commonSituations":"Attention implementations that transpose (b, h, s, s) score tensors before softmax on Apple GPUs; narrowed sequence windows; strided views from cache lookups.","solutions":["Insert .contiguous() before softmax_last_dim","Restructure so softmax runs before the transpose (softmax over last dim of the contiguous tensor)","Check layout contiguity in debug builds during development to catch it early"],"exampleFix":"// before\nlet probs = softmax_last_dim(&scores.transpose(1, 2)?)?;\n// after\nlet probs = softmax_last_dim(&scores.transpose(1, 2)?.contiguous()?)?;","handlingStrategy":"validation","validationCode":"let t = if t.layout().is_contiguous() { t } else { t.contiguous()? };\nlet probs = softmax_last_dim(&t)?;","typeGuard":"fn is_contiguous(t: &Tensor) -> bool { t.layout().contiguous_offsets().is_some() }","tryCatchPattern":"match softmax_last_dim(&t) {\n    Ok(v) => v,\n    Err(e) if e.to_string().contains(\"contiguous\") => softmax_last_dim(&t.contiguous()?)?,\n    Err(e) => return Err(e),\n}","preventionTips":["Materialize views with .contiguous() before Metal kernels","Run softmax on the pre-transpose contiguous tensor","Assert contiguity in tests when targeting Apple GPUs"],"tags":["metal","softmax","contiguity","gpu","candle"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}