{"record":{"id":"31070eef9df6dbb9","repo":"sgl-project/sglang","slug":"sparse-attn-v4-paged-prefill-expects-fp16-bf16-q","errorCode":null,"errorMessage":"sparse_attn_v4_paged_prefill expects fp16/bf16 q, got {q.dtype}","messagePattern":"sparse_attn_v4_paged_prefill expects fp16/bf16 q, got (.+?)","errorType":"validation","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/dsv4/unified_kv_kernels/paged_prefill.py","lineNumber":233,"sourceCode":"\n\ndef _sparse_attn_v4_paged_prefill_triton(\n    q: torch.Tensor,\n    unified_kv: torch.Tensor,\n    kv_indices_prefix: torch.Tensor,\n    kv_indptr_prefix: torch.Tensor,\n    kv: torch.Tensor,\n    kv_indices_extend: torch.Tensor,\n    kv_indptr_extend: torch.Tensor,\n    attn_sink: torch.Tensor,\n    softmax_scale: float,\n) -> torch.Tensor:\n    if not q.is_cuda:\n        raise RuntimeError(\n            \"Triton sparse_attn_v4_paged_prefill requires CUDA/HIP tensors\"\n        )\n    if q.dtype not in (torch.bfloat16, torch.float16):\n        raise RuntimeError(\n            f\"sparse_attn_v4_paged_prefill expects fp16/bf16 q, got {q.dtype}\"\n        )\n    if unified_kv.dtype != q.dtype:\n        raise RuntimeError(\n            f\"unified_kv dtype mismatch: kv={unified_kv.dtype}, q={q.dtype}\"\n        )\n    if kv.dtype != q.dtype:\n        raise RuntimeError(f\"kv dtype mismatch: kv={kv.dtype}, q={q.dtype}\")\n    if unified_kv.size(-1) != kv.size(-1):\n        raise RuntimeError(\n            f\"head_dim mismatch: unified_kv={unified_kv.size(-1)}, kv={kv.size(-1)}\"\n        )\n\n    T, H, D = q.shape\n    out = torch.empty_like(q)\n    kv_indices_prefix = kv_indices_prefix.to(torch.int32).contiguous()\n    kv_indptr_prefix = kv_indptr_prefix.to(torch.int32).contiguous()\n    kv_indices_extend = kv_indices_extend.to(torch.int32).contiguous()","sourceCodeStart":215,"sourceCodeEnd":251,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/dsv4/unified_kv_kernels/paged_prefill.py#L215-L251","documentation":"The Triton sparse_attn_v4_paged_prefill kernel only supports fp16/bf16 queries; any other q dtype is rejected before kernel launch because the Triton code is specialized for 16-bit math.","triggerScenarios":"Calling sparse_attn_v4_paged_prefill with q in float32 (or fp8) dtype.","commonSituations":"Prefill fixtures in fp32 in tests; a model configuration leaving hidden states in fp32; a --dtype float32 run; passing a logits-scale or normalized tensor of the wrong dtype.","solutions":["Cast q to torch.bfloat16 or torch.float16 before the call","Align the model/dtype configuration with a supported 16-bit dtype","Add an early dtype assert at the call site to catch the producer"],"exampleFix":"// before\nout = sparse_attn_v4_paged_prefill(q_fp32, ...)\n// after\nout = sparse_attn_v4_paged_prefill(q_fp32.to(torch.bfloat16), ...)","handlingStrategy":"validation","validationCode":"if q.dtype not in (torch.float16, torch.bfloat16):\n    q = q.to(torch.bfloat16)","typeGuard":"def q_dtype_ok(q: torch.Tensor) -> bool:\n    return q.dtype in (torch.float16, torch.bfloat16)","tryCatchPattern":null,"preventionTips":["Keep a shared dtype guard helper for all dsv4 attention call sites","Avoid --dtype float32 for models routed to these Triton kernels"],"tags":["attention","dtype","triton","prefill"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}