What causes "Non contiguous softmax-last-dim is not implemented" in huggingface/candle?

Insert .contiguous() before softmax_last_dim

Non contiguous softmax-last-dim is not implemented

Error message

Non contiguous softmax-last-dim is not implemented

What it means

The Metal softmax kernel requires the tensor to be contiguous with the last dimension having stride 1. Non-contiguous views (from transpose/permute/narrow) are not supported by the shader, so metal_fwd bails explicitly rather than producing wrong results.

Source

Thrown at candle-nn/src/ops.rs:409

        &self,
        storage: &candle::MetalStorage,
        layout: &Layout,
    ) -> Result<(candle::MetalStorage, Shape)> {
        use candle::backend::BackendStorage;
        let device = storage.device();
        let encoder = device.command_encoder()?;
        encoder.set_label("softmax");
        let kernels = device.kernels();
        let name = match storage.dtype() {
            DType::F32 => "softmax_f32",
            DType::F16 => "softmax_f16",
            DType::BF16 => "softmax_bf16",
            dtype => candle::bail!("softmax-last-dim is not implemented for {dtype:?}"),
        };

        let n = layout.stride().len();
        if !(layout.is_contiguous() && layout.stride()[n - 1] == 1) {
            candle::bail!("Non contiguous softmax-last-dim is not implemented");
        }

        let last_dim = layout.dims()[layout.shape().rank() - 1];
        let elem_count = layout.shape().elem_count();
        let output = device
            .new_buffer_builder()
            .with_size_for(elem_count, storage.dtype())
            .with_label("softmax")
            .build()?;
        candle_metal_kernels::call_last_softmax(
            device.metal_device(),
            &encoder,
            kernels,
            name,
            elem_count,
            last_dim,
            storage.buffer(),
            layout.start_offset() * storage.dtype().size_in_bytes(),

View on GitHub (pinned to d5fee525bf)

Solutions

Insert .contiguous() before softmax_last_dim
Restructure so softmax runs before the transpose (softmax over last dim of the contiguous tensor)
Check layout contiguity in debug builds during development to catch it early

Example fix

// before
let probs = softmax_last_dim(&scores.transpose(1, 2)?)?;
// after
let probs = softmax_last_dim(&scores.transpose(1, 2)?.contiguous()?)?;

Defensive patterns

Strategy: validation

Validate before calling

let t = if t.layout().is_contiguous() { t } else { t.contiguous()? };
let probs = softmax_last_dim(&t)?;

Type guard

fn is_contiguous(t: &Tensor) -> bool { t.layout().contiguous_offsets().is_some() }

Try / catch

match softmax_last_dim(&t) {
    Ok(v) => v,
    Err(e) if e.to_string().contains("contiguous") => softmax_last_dim(&t.contiguous()?)?,
    Err(e) => return Err(e),
}

Prevention

Materialize views with .contiguous() before Metal kernels
Run softmax on the pre-transpose contiguous tensor
Assert contiguity in tests when targeting Apple GPUs

When it happens

Trigger: Calling ops::softmax_last_dim on a Metal tensor whose layout fails layout.is_contiguous() && stride()[last]==1 — typically after transpose/permute/slice.

Common situations: Attention implementations that transpose (b, h, s, s) score tensors before softmax on Apple GPUs; narrowed sequence windows; strided views from cache lookups.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/017a7615ae46d414. Report an issue: GitHub.