What causes "input2 has to be contiguous" in huggingface/candle?

Insert `.contiguous()?` on the second input before the op.

input2 has to be contiguous

Error message

input2 has to be contiguous

What it means

Same custom Add3 op in the SAM image encoder, but for the second input. The CPU kernel reads raw f32 slices via `contiguous_offsets()` on layout l2; if the second tensor's layout is not a single contiguous range it bails to avoid reading wrong data.

Source

Thrown at candle-transformers/src/models/segment_anything/image_encoder.rs:69

        &self,
        s1: &candle::CpuStorage,
        l1: &candle::Layout,
        s2: &candle::CpuStorage,
        l2: &candle::Layout,
        s3: &candle::CpuStorage,
        l3: &candle::Layout,
    ) -> Result<(candle::CpuStorage, candle::Shape)> {
        use rayon::prelude::*;

        let Add3(b, q_h, q_w, k_h, k_w) = *self;
        let s1 = s1.as_slice::<f32>()?;
        let s1 = match l1.contiguous_offsets() {
            None => candle::bail!("input1 has to be contiguous"),
            Some((o1, o2)) => &s1[o1..o2],
        };
        let s2 = s2.as_slice::<f32>()?;
        let s2 = match l2.contiguous_offsets() {
            None => candle::bail!("input2 has to be contiguous"),
            Some((o1, o2)) => &s2[o1..o2],
        };
        let s3 = s3.as_slice::<f32>()?;
        let s3 = match l3.contiguous_offsets() {
            None => candle::bail!("input3 has to be contiguous"),
            Some((o1, o2)) => &s3[o1..o2],
        };
        let mut dst = vec![0f32; b * q_h * q_w * k_h * k_w];
        dst.par_chunks_exact_mut(k_h * k_w)
            .enumerate()
            .for_each(|(b_idx, dst)| {
                let s1_idx = b_idx * k_h * k_w;
                let s2_idx = b_idx * k_h;
                let s3_idx = b_idx * k_w;
                for h_idx in 0..k_h {
                    let s1_idx = s1_idx + h_idx * k_w;
                    let s2_idx = s2_idx + h_idx;
                    let dst_idx = h_idx * k_w;

View on GitHub (pinned to d5fee525bf)

Solutions

Insert `.contiguous()?` on the second input before the op.
Match the pattern used for k in this file: `k.squeeze(0)?.transpose(0, 1)?.contiguous()?`.
If this fires inside library code you didn't modify, check you're calling the public encoder API rather than the raw custom op.
Update candle — later versions may add strided handling.

Example fix

// before
let k = keys.transpose(0, 1);
let out = add3(q, k, v)?;
// after
let out = add3(q, keys.transpose(0, 1)?.contiguous()?, v)?;

Defensive patterns

Strategy: validation

Validate before calling

fn ensure_contiguous(t: &Tensor) -> candle::Result<Tensor> {
    if t.layout().contiguous_offsets().is_none() { t.contiguous() } else { Ok(t.clone()) }
}
let k = ensure_contiguous(&input2)?;

Type guard

fn is_contiguous(t: &Tensor) -> bool {
    t.layout().contiguous_offsets().is_some()
}

Try / catch

match add3_forward(q, k, v) {
    Ok(t) => t,
    Err(e) if e.to_string().contains("input2 has to be contiguous") => {
        add3_forward(q, k.contiguous()?, v)
    }
    Err(e) => return Err(e.into()),
}

Prevention

Materialize transposed tensors with .contiguous() before feeding custom ops.
Follow the in-file idiom: tensor.transpose(0,1)?.contiguous()?.
Avoid passing broadcast views directly into Add3-style kernels.
Add assertions on contiguity at model boundaries during development.

When it happens

Trigger: The second argument to the add-by-broadcast op (K-side attention tensor) is non-contiguous — usually the output of `transpose`/`permute`/`narrow` passed straight into the encoder's attention add without `.contiguous()`.

Common situations: Arises when refactoring the attention computation to fuse reshapes, or when porting PyTorch code where views are lazily strided but this kernel requires materialized memory.

Related errors

AI-assisted analysis of huggingface/candle@d5fee525bf (2026-09-02). Data as JSON: /api/errors/8b8441dcf48aefa4. Report an issue: GitHub.