{"record":{"id":"e7d305f77aed5235","repo":"huggingface/candle","slug":"seqlens-q-has-to-be-contiguous","errorCode":null,"errorMessage":"seqlens_q has to be contiguous","messagePattern":"seqlens_q has to be contiguous","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn-v3/src/lib.rs","lineNumber":468,"sourceCode":"        k: &candle::CudaStorage,\n        k_l: &Layout,\n        v: &candle::CudaStorage,\n        v_l: &Layout,\n        is_bf16: bool,\n    ) -> Result<(candle::CudaStorage, Shape)> {\n        // https://github.com/Dao-AILab/flash-attention/blob/0dfb28174333d9eefb7c1dd4292690a8458d1e89/hopper/flash_api.cpp\n        let dev = q.device();\n        let out_shape = q_l.shape().clone();\n        let out_l = Layout::contiguous(&out_shape);\n\n        let (seqlens_q, seqlens_q_layout) = self.seqlens_q.storage_and_layout();\n        let seqlens_q = match &*seqlens_q {\n            candle::Storage::Cuda(c) => c.as_cuda_slice::<u32>()?, // Should be i32!\n            _ => candle::bail!(\"seqlens_q must be a cuda tensor\"),\n        };\n        let seqlens_q = match seqlens_q_layout.contiguous_offsets() {\n            Some((o1, o2)) => seqlens_q.slice(o1..o2),\n            None => candle::bail!(\"seqlens_q has to be contiguous\"),\n        };\n\n        let (seqlens_k, seqlens_k_layout) = self.seqlens_k.storage_and_layout();\n        let seqlens_k = match &*seqlens_k {\n            candle::Storage::Cuda(c) => c.as_cuda_slice::<u32>()?, // Should be i32!\n            _ => candle::bail!(\"seqlens_k must be a cuda tensor\"),\n        };\n        let seqlens_k = match seqlens_k_layout.contiguous_offsets() {\n            Some((o1, o2)) => seqlens_k.slice(o1..o2),\n            None => candle::bail!(\"seqlens_k has to be contiguous\"),\n        };\n\n        let q = q.as_cuda_slice::<T>()?;\n        let k = k.as_cuda_slice::<T>()?;\n        let v = v.as_cuda_slice::<T>()?;\n        let q = q.slice(q_l.start_offset()..);\n        let k = k.slice(k_l.start_offset()..);\n        let v = v.slice(v_l.start_offset()..);","sourceCodeStart":450,"sourceCodeEnd":486,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn-v3/src/lib.rs#L450-L486","documentation":"The varlen forward requires seqlens_q (cu_seqlens_q, cumulative query-sequence offsets) to occupy a single contiguous memory range, because it is converted into a raw CUDA slice via contiguous_offsets(). The check returned None, so the tensor's layout is strided or a view not expressible as one contiguous run. The library bails rather than silently misreading offsets on the GPU.","triggerScenarios":"Calling the varlen forward (cuda_fwd_t path) with seqlens_q produced by a strided slice of a larger tensor, a transpose/permute view, or chained narrow/index_select ops leaving a non-contiguous layout.","commonSituations":"Slicing cu_seqlens out of a combined offsets buffer; building offsets on CPU and moving only a sub-view to CUDA; reusing metadata tensors reshaped for other kernels without materializing a copy.","solutions":["Call .contiguous() on seqlens_q before passing it to the forward call.","Create seqlens_q fresh as a 1-D contiguous u32 CUDA tensor (Tensor::from_vec on the device) instead of slicing.","Insert .contiguous() at the pipeline boundary where offsets are handed to the attention module."],"exampleFix":"// before\nlet seqlens_q = offsets.i(1..)?; // strided view\nflash_attn_varlen_forward(&q, &k, &v, &seqlens_q, &seqlens_k, ...)?;\n// after\nlet seqlens_q = offsets.i(1..)?.to_device(q.device())?.contiguous()?;\nflash_attn_varlen_forward(&q, &k, &v, &seqlens_q, &seqlens_k, ...)?;","handlingStrategy":"validation","validationCode":"fn check_seqlens_q(t: &candle_core::Tensor) -> candle_core::Result<()> {\n    if t.device().is_cpu() { candle_core::bail!(\"seqlens_q must be on CUDA\"); }\n    if t.dtype() != candle_core::DType::U32 { candle_core::bail!(\"seqlens_q must be u32\"); }\n    if t.contiguous_offsets().is_none() { candle_core::bail!(\"seqlens_q not contiguous; call .contiguous()\"); }\n    Ok(())\n}\n// run before the forward call: check_seqlens_q(&seqlens_q)?;","typeGuard":"fn is_contiguous_cuda(t: &candle_core::Tensor) -> bool {\n    !t.device().is_cpu() && t.contiguous_offsets().is_some()\n}","tryCatchPattern":"match flash_attn_varlen_forward(&q, &k, &v, &seqlens_q, &seqlens_k, ...) {\n    Ok(out) => out,\n    Err(e) if e.to_string().contains(\"seqlens_q has to be contiguous\") => {\n        let sq = seqlens_q.contiguous()?;\n        flash_attn_varlen_forward(&q, &k, &v, &sq, &seqlens_k, ...)?\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Always call .contiguous() on any cu_seqlens tensor right before the flash-attn varlen call.","Create cu_seqlens fresh via Tensor::from_vec on the CUDA device instead of slicing existing buffers.","Add a debug assertion checking contiguous_offsets().is_some() at the attention module boundary."],"tags":["cuda","tensor-layout","contiguity","flash-attention"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}