{"record":{"id":"acc018ef5e4f360c","repo":"sgl-project/sglang","slug":"type-mismatch-self-q-dtype-self-v-dtype","errorCode":null,"errorMessage":"Type mismatch: {self.q_dtype} != {self.v_dtype}","messagePattern":"Type mismatch: (.+?) != (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_sm100.py","lineNumber":870,"sourceCode":"            if const_expr(self.kv_sf_interleaved):\n                mSFV = cute.make_tensor(mSFV.iterator, sfv_layout)\n            else:\n                assert (\n                    not self.use_tma_KV\n                ), \"can't use TMA to load SFV if not interleaved in gmem\"\n                mSFV = cute.make_tensor(\n                    mSFV.iterator, cute.select(mSFV.layout, mode=KV_layout_transpose)\n                )\n\n        # check type consistency\n        if const_expr(self.q_dtype != self.k_dtype):\n            raise TypeError(f\"Type mismatch: {self.q_dtype} != {self.k_dtype}\")\n        if const_expr(\n            not self.qk_blockscaled\n            and not self.v_dequant\n            and self.q_dtype != self.v_dtype\n        ):\n            raise TypeError(f\"Type mismatch: {self.q_dtype} != {self.v_dtype}\")\n        if const_expr(self.qk_blockscaled and self.sfq_dtype != self.sfk_dtype):\n            raise TypeError(f\"Type mismatch: {self.sfq_dtype} != {self.sfk_dtype}\")\n        if const_expr(self.q_dtype.width == 8):\n            paged_kv_non_tma = not self.use_tma_KV\n            if const_expr(self.head_dim_padded < 96):\n                fp8_regs = _FP8_SMALL_HDIM_REGS[paged_kv_non_tma]\n                self.num_regs_softmax = fp8_regs[\"num_regs_softmax\"]\n                self.num_regs_correction = fp8_regs[\"num_regs_correction\"]\n                self.num_regs_other = fp8_regs[\"num_regs_other\"]\n            else:\n                fp8_tune = _FP8_TUNING_CONFIG.get(\n                    (\n                        self.use_2cta_instrs,\n                        self.is_causal,\n                        self.head_dim_padded,\n                        self.is_sm103,\n                    ),\n                    {},","sourceCodeStart":852,"sourceCodeEnd":888,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_sm100.py#L852-L888","documentation":"Unless the kernel is block-scaled (qk_blockscaled) or V is dequantized in-kernel (v_dequant), V must share Q's dtype. This check is skipped when V is fp8 with separate scale factors (v_blockscaled path), where bf16 output is expected.","triggerScenarios":"Calling the SM100 kernel with q_dtype != v_dtype while neither qk_blockscaled nor v_dequant is enabled — e.g. bf16 Q/K with fp8 V and no SFV/scale configuration.","commonSituations":"Trying to save memory by quantizing only V to fp8 without wiring up the block-scaled (scale factor) path.","solutions":["If quantizing V to fp8, enable the v_blockscaled/v_dequant path with sfv scale factors","Otherwise cast V to match Q's dtype (bf16)"],"exampleFix":"// before\nattn(q_bf16, k_bf16, v_fp8)  # no sfv\n// after\nv = v.to(torch.bfloat16)\nattn(q_bf16, k_bf16, v)","handlingStrategy":"validation","validationCode":"uses_sf = v_blockscaled or v_dequant\nassert uses_sf or q.dtype == v.dtype","typeGuard":"def v_dtype_ok(q, v, v_blockscaled=False, v_dequant=False) -> bool:\n    return v_blockscaled or v_dequant or q.dtype == v.dtype","tryCatchPattern":null,"preventionTips":["If quantizing V to fp8, wire the scale-factor (sfv) path rather than raw fp8 V","Default to bf16 everywhere unless the whole fp8 recipe is set up"],"tags":["flash-attention","dtype-mismatch","fp8","sm100"],"backgroundTag":"dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}