{"record":{"id":"e09cee8579ae3b53","repo":"huggingface/candle","slug":"seqlens-k-has-to-be-contiguous","errorCode":null,"errorMessage":"seqlens_k has to be contiguous","messagePattern":"seqlens_k has to be contiguous","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn-v3/src/lib.rs","lineNumber":478,"sourceCode":"\n        let (seqlens_q, seqlens_q_layout) = self.seqlens_q.storage_and_layout();\n        let seqlens_q = match &*seqlens_q {\n            candle::Storage::Cuda(c) => c.as_cuda_slice::<u32>()?, // Should be i32!\n            _ => candle::bail!(\"seqlens_q must be a cuda tensor\"),\n        };\n        let seqlens_q = match seqlens_q_layout.contiguous_offsets() {\n            Some((o1, o2)) => seqlens_q.slice(o1..o2),\n            None => candle::bail!(\"seqlens_q has to be contiguous\"),\n        };\n\n        let (seqlens_k, seqlens_k_layout) = self.seqlens_k.storage_and_layout();\n        let seqlens_k = match &*seqlens_k {\n            candle::Storage::Cuda(c) => c.as_cuda_slice::<u32>()?, // Should be i32!\n            _ => candle::bail!(\"seqlens_k must be a cuda tensor\"),\n        };\n        let seqlens_k = match seqlens_k_layout.contiguous_offsets() {\n            Some((o1, o2)) => seqlens_k.slice(o1..o2),\n            None => candle::bail!(\"seqlens_k has to be contiguous\"),\n        };\n\n        let q = q.as_cuda_slice::<T>()?;\n        let k = k.as_cuda_slice::<T>()?;\n        let v = v.as_cuda_slice::<T>()?;\n        let q = q.slice(q_l.start_offset()..);\n        let k = k.slice(k_l.start_offset()..);\n        let v = v.slice(v_l.start_offset()..);\n\n        let q_stride = q_l.stride();\n        let k_stride = k_l.stride();\n        let v_stride = v_l.stride();\n        let o_stride = out_l.stride();\n\n        let q_rank = q_stride.len();\n        let k_rank = k_stride.len();\n        let v_rank = v_stride.len();\n        let o_rank = o_stride.len();","sourceCodeStart":460,"sourceCodeEnd":496,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn-v3/src/lib.rs#L460-L496","documentation":"Same contiguity precondition as seqlens_q, applied to seqlens_k: the tensor must map to a single contiguous offset range so it can be turned into a raw CUDA slice for the kernel. contiguous_offsets() returned None because the layout is strided or otherwise not contiguous.","triggerScenarios":"Passing a non-contiguous seqlens_k (strided slice, permuted view, or narrow of a larger offsets tensor) into the varlen forward's cuda_fwd_t path.","commonSituations":"Slicing cu_seqlens_k out of a shared offsets buffer; deriving offsets via ops that keep non-unit strides; reusing a transposed metadata tensor across kernels.","solutions":["Call .contiguous() on seqlens_k before the forward call.","Create seqlens_k as a fresh 1-D tensor on the CUDA device rather than a view of a larger buffer.","Materialize derived offsets, e.g. tensor.flatten_all()?.contiguous()?."],"exampleFix":"// before\nlet seqlens_k = combined_offsets.i(..)?; // non-contiguous view\n// after\nlet seqlens_k = combined_offsets.i(..)?.contiguous()?;","handlingStrategy":"validation","validationCode":"let seqlens_k = if seqlens_k.contiguous_offsets().is_none() {\n    seqlens_k.contiguous()?\n} else { seqlens_k };","typeGuard":"fn is_contiguous(t: &candle_core::Tensor) -> bool { t.contiguous_offsets().is_some() }","tryCatchPattern":"match forward(&q, &k, &v, &seqlens_q, &seqlens_k, ...) {\n    Err(e) if e.to_string().contains(\"seqlens_k has to be contiguous\") => {\n        let sk = seqlens_k.contiguous()?;\n        forward(&q, &k, &v, &seqlens_q, &sk, ...)\n    }\n    other => other,\n}","preventionTips":["Materialize sliced or derived offsets with .contiguous() before the kernel call.","Keep cu_seqlens_k as a standalone 1-D tensor rather than a view of a larger buffer.","Test the attention path with tensors produced by the real preprocessing pipeline, not just fresh contiguous ones."],"tags":["cuda","tensor-layout","contiguity","flash-attention"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}