{"record":{"id":"263d2e004a73e9f7","repo":"tracel-ai/burn","slug":"attention-unsupported-dtype","errorCode":null,"errorMessage":"attention: unsupported dtype {:?}","messagePattern":"attention: unsupported dtype (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/burn-flex/src/ops/attention.rs","lineNumber":114,"sourceCode":"                    mask,\n                    attn_bias.map(|b| cast_to_f32(b, f16::to_f32)),\n                    options,\n                );\n                cast_from_f32(r, f16::from_f32)\n            }\n            DType::BF16 => {\n                use burn_std::bf16;\n                let r = $impl_fn::<f32>(\n                    cast_to_f32(query, bf16::to_f32),\n                    cast_to_f32(key, bf16::to_f32),\n                    cast_to_f32(value, bf16::to_f32),\n                    mask,\n                    attn_bias.map(|b| cast_to_f32(b, bf16::to_f32)),\n                    options,\n                );\n                cast_from_f32(r, bf16::from_f32)\n            }\n            dtype => panic!(\"attention: unsupported dtype {:?}\", dtype),\n        }\n    }};\n}\n\n/// Contiguous mask/bias tensor plus the per-batch and per-head element offsets the\n/// inner loop should use to locate the `[seq_q, seq_kv]` tile for each `(batch, head)`\n/// pair. When a leading dim (batch or heads) is `1` in the source, its step is `0`, so\n/// the inner loop re-reads the same tile for every pair without allocating an expanded\n/// copy. The tile length itself is always `seq_q * seq_kv` and is computed at the call\n/// site, so it is not stored here.\nstruct BroadcastMaskBias {\n    tensor: FlexTensor,\n    batch_step: usize,\n    head_step: usize,\n}\n\n/// Prepare an attention mask or bias for the inner loop, accepting ONNX Attention-23\n/// broadcast shapes.","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/tracel-ai/burn/blob/d16f7ba2ed0d41408189384044cc886fb4c8f957/crates/burn-flex/src/ops/attention.rs#L96-L132","documentation":"The flex attention kernel macro handles F32, F64, F16 and BF16 (with half-precision paths casting mask/bias to f32), and panics with \"attention: unsupported dtype\" for any other query/key dtype. Attention math is inherently floating-point, so integer/bool inputs are rejected deliberately.","triggerScenarios":"Invoking the flex attention op with query/key/value tensors of an integer or bool dtype. Note the DECLARED-AS hint: a dtype value flowing through burn-autodiff's supports_dtype/dtype_usage hook surface (crates/burn-autodiff/src/backend.rs:100) that was accepted upstream can still reach this kernel unhandled if the inner backend's dtype filtering and the kernel's match disagree.","commonSituations":"Token ids (integer embeddings indices) passed straight to attention instead of after the embedding lookup; a custom dtype-usage hook allowing a dtype the flex kernel lacks; backend version drift where autodiff forwards dtypes the flex impl never added.","solutions":["Cast q/k/v to F32 (or F16/BF16) before the attention call: q.cast(DType::F32) etc.","Verify tokens go through the embedding layer so attention receives float tensors, not token indices.","If a hook (new_with_hook / dtype_usage) advertises an unsupported dtype, restrict supported_dtype to F32/F64/F16/BF16 on the backend."],"exampleFix":"// before\nlet attn = attention(q_ids, k_ids, v, mask, options); // integer\n// after\nlet attn = attention(q_ids.cast(DType::F32), k_ids.cast(DType::F32), v.cast(DType::F32), mask, options);","handlingStrategy":"validation","validationCode":"for t in [&q, &k, &v] {\n    if !matches!(t.dtype(), DType::F32 | DType::F64 | DType::F16 | DType::BF16) {\n        panic!(\"attention inputs must be float, got {:?}\", t.dtype());\n    }\n}","typeGuard":"fn is_float_tensor(t: &FlexTensor) -> bool {\n    matches!(t.dtype(), DType::F32 | DType::F64 | DType::F16 | DType::BF16)\n}","tryCatchPattern":"let out = std::panic::catch_unwind(|| attention(q.clone(), k.clone(), v.clone(), mask, options))\n    .unwrap_or_else(|_| attention(q.cast(DType::F32), k.cast(DType::F32), v.cast(DType::F32), mask, options));","preventionTips":["Never pass raw token ids to attention; route them through embeddings first.","Restrict backend supports_dtype to the dtypes the kernel actually implements.","Test attention with each dtype the model config allows.","Validate dtype agreement across q/k/v and mask."],"tags":["panic","dtype","attention","unsupported-dtype"],"backgroundTag":"unsupported-dtype","analyzedSha":"d16f7ba2ed0d41408189384044cc886fb4c8f957","analyzedAt":"2026-09-05T13:19:14.260Z","contentChangedAt":"2026-09-05T13:19:14.260Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}