{"record":{"id":"7103d96fa2862e2c","repo":"huggingface/candle","slug":"shape-mismatch-alibi-slopes-expected-7103d9","errorCode":null,"errorMessage":"shape mismatch alibi_slopes {:?}, expected {:?}","messagePattern":"shape mismatch alibi_slopes (.+?), expected (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"candle-flash-attn/src/lib.rs","lineNumber":103,"sourceCode":"        }\n        if num_heads % num_heads_k != 0 {\n            candle::bail!(\"number of k/v heads {num_heads_k} must divide number of heads in query {num_heads}\")\n        }\n\n        let stream = dev.cuda_stream();\n        let alibi_slopes_ptr = if let Some(alibi_slopes) = &self.alibi_slopes {\n            if alibi_slopes.dtype() != DType::F32 {\n                candle::bail!(\n                    \"DType mismatch alibi_slopes {:?}, expected {:?}\",\n                    alibi_slopes.dtype(),\n                    DType::F32\n                );\n            }\n\n            let (alibi_slopes, alibi_slopes_layout) = alibi_slopes.storage_and_layout();\n\n            if num_heads != alibi_slopes_layout.shape().dims1()? {\n                candle::bail!(\n                    \"shape mismatch alibi_slopes {:?}, expected {:?}\",\n                    alibi_slopes_layout.shape(),\n                    (num_heads)\n                );\n            }\n\n            let alibi_slopes = match &*alibi_slopes {\n                candle::Storage::Cuda(c) => c.as_cuda_slice::<f32>()?,\n                _ => candle::bail!(\"alibi_slopes must be a cuda tensor\"),\n            };\n\n            let alibi_slopes = alibi_slopes.slice(alibi_slopes_layout.start_offset()..);\n\n            // Dropping the guard here doesn't seem very safe.\n            let (ptr, _guard) = alibi_slopes.device_ptr(&stream);\n            ptr as *const core::ffi::c_void\n        } else {\n            std::ptr::null()","sourceCodeStart":85,"sourceCodeEnd":121,"githubUrl":"https://github.com/huggingface/candle/blob/d5fee525bfde3273eb7c9b75fd2bc4937be867ca/candle-flash-attn/src/lib.rs#L85-L121","documentation":"candle-flash-attn throws this when the optional alibi_slopes tensor's shape does not have exactly `num_heads` elements. ALIBI slopes must provide one slope value per attention head, so the kernel validates dim1 size == num_heads before launch.","triggerScenarios":"Calling flash_attn (or FlashAttnV2 forward) with an `alibi_slopes` tensor whose rank is not 1 or whose length differs from the q tensor's num_heads dimension; `dims1()?` also fails for non-1D tensors and that error propagates similarly.","commonSituations":"Reusing slopes prepared for a different head count after changing model config (e.g. num_attention_heads), passing a [1, num_heads] or [num_heads, 1] tensor instead of 1D, or passing a wrong slice of a larger slopes tensor.","solutions":["Reshape alibi_slopes to a 1D tensor of exactly num_heads elements before calling","Regenerate slopes for the current num_heads config (slopes depend on head count)","Remove the alibi_slopes argument if ALIBI is not intended"],"exampleFix":"// before\nlet slopes = Tensor::from_vec(slopes_vec, (1, num_heads), &dev)?;\nflash_attn(&q, &k, &v, Some(&slopes), softmax_scale, causal)?\n// after\nlet slopes = Tensor::from_vec(slopes_vec, (num_heads,), &dev)?;\nflash_attn(&q, &k, &v, Some(&slopes), softmax_scale, causal)?","handlingStrategy":"validation","validationCode":"fn check_alibi(slopes: &Tensor, num_heads: usize) -> candle::Result<()> {\n    let dims = slopes.dims1()?; // errors if not 1D\n    if dims != num_heads {\n        candle::bail!(\"alibi_slopes len {dims} != num_heads {num_heads}\");\n    }\n    Ok(())\n}","typeGuard":"fn is_valid_alibi(slopes: &Tensor, num_heads: usize) -> bool {\n    slopes.dims() == [num_heads]\n}","tryCatchPattern":"match flash_attn(&q, &k, &v, Some(&slopes), scale, causal) {\n    Err(candle::Error::Msg(m)) if m.contains(\"alibi_slopes\") => { /* rebuild slopes with correct num_heads */ }\n    r => r?,\n}","preventionTips":["Derive alibi slopes programmatically from num_heads instead of hardcoding","Assert slopes.dim(0) == q.dim(2) (num_heads) in model init","Keep slopes 1D; never add batch/head axes"],"tags":["cuda","flash-attention","shape-mismatch","tensor-shape"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"d5fee525bfde3273eb7c9b75fd2bc4937be867ca","analyzedAt":"2026-09-02T00:15:47.023Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-09T06:17:21.866Z"}