huggingface/candle · error

mm_prefix_ranges last dimension must be contiguous

Error message

mm_prefix_ranges last dimension must be contiguous

What it means

After the shape check, the kernel requires mm_prefix_ranges to have its last dimension contiguous (stride 1), because it reads range pairs as flat (start,end) int pairs via raw pointers. Non-contiguous last dims (from slicing, transposing, or strided views) would make the kernel read wrong memory.

Source

Thrown at candle-flash-attn/src/lib.rs:638

            if mm_prefix_ranges.dtype() != DType::I32 {
                candle::bail!(
                    "mm_prefix_ranges must be i32, got {:?}",
                    mm_prefix_ranges.dtype()
                )
            }
            match &*storage {
                candle::Storage::Cuda(_) => {}
                _ => candle::bail!("mm_prefix_ranges must be a cuda tensor"),
            }
            let (mm_batch, max_ranges, two) = layout.shape().dims3()?;
            if mm_batch != batch_size || two != 2 {
                candle::bail!(
                    "mm_prefix_ranges shape must be ({batch_size}, max_ranges, 2), got {:?}",
                    layout.shape()
                )
            }
            if layout.stride().last().copied() != Some(1) {
                candle::bail!("mm_prefix_ranges last dimension must be contiguous")
            }
            Some((
                storage,
                layout.start_offset(),
                layout.stride()[0],
                max_ranges,
            ))
        } else {
            None
        };

        let stream = dev.cuda_stream();
        let alibi_slopes_ptr = if let Some(alibi_slopes) = &self.alibi_slopes {
            if alibi_slopes.dtype() != DType::F32 {
                candle::bail!(
                    "DType mismatch alibi_slopes {:?}, expected {:?}",
                    alibi_slopes.dtype(),
                    DType::F32

View on GitHub (pinned to d5fee525bf)

Solutions

1. Call .contiguous() on mm_prefix_ranges before passing it to flash-attn
2. Construct the tensor in (batch, max_ranges, 2) layout directly instead of transposing
3. Avoid narrow/slice views on the last dimension; copy into a fresh contiguous tensor

Example fix

// before
let ranges = ranges.t()?.contiguous()?; // still strided last dim in some views
// after
let ranges = ranges.t()?.contiguous()?.contiguous(); // ensure C-contiguous
// or simply:
let ranges = ranges.to_contiguous()?;

let ranges = ranges.contiguous()?; // ensure last-dim stride == 1 before the call

match result {
    Err(e) if e.to_string().contains("last dimension must be contiguous") => {
        let r = ranges.contiguous()?;
        // retry with contiguous tensor
    }
    other => other?,
}

When it happens

Trigger: Passing an mm_prefix_ranges tensor produced by strided slicing/transposing/permute without calling .contiguous(), so layout.stride().last() != Some(1).

Common situations: Building ranges from a larger buffer via narrow/slice, or transposing a (batch, 2, max_ranges) tensor into (batch, max_ranges, 2) without materializing it.

Related errors

• input has to be contiguous
• quantized embedding requires contiguous ids
• seqlens_q has to be contiguous
• seqlens_k has to be contiguous
• seqlens_q has to be contiguous

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/94985b9da748d903. Report an issue: GitHub.