{"record":{"id":"0fe17dc7b6e2e44e","repo":"sgl-project/sglang","slug":"type-mismatch-self-sfq-dtype-self-sfk-dtype","errorCode":null,"errorMessage":"Type mismatch: {self.sfq_dtype} != {self.sfk_dtype}","messagePattern":"Type mismatch: (.+?) != (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_sm100.py","lineNumber":872,"sourceCode":"            else:\n                assert (\n                    not self.use_tma_KV\n                ), \"can't use TMA to load SFV if not interleaved in gmem\"\n                mSFV = cute.make_tensor(\n                    mSFV.iterator, cute.select(mSFV.layout, mode=KV_layout_transpose)\n                )\n\n        # check type consistency\n        if const_expr(self.q_dtype != self.k_dtype):\n            raise TypeError(f\"Type mismatch: {self.q_dtype} != {self.k_dtype}\")\n        if const_expr(\n            not self.qk_blockscaled\n            and not self.v_dequant\n            and self.q_dtype != self.v_dtype\n        ):\n            raise TypeError(f\"Type mismatch: {self.q_dtype} != {self.v_dtype}\")\n        if const_expr(self.qk_blockscaled and self.sfq_dtype != self.sfk_dtype):\n            raise TypeError(f\"Type mismatch: {self.sfq_dtype} != {self.sfk_dtype}\")\n        if const_expr(self.q_dtype.width == 8):\n            paged_kv_non_tma = not self.use_tma_KV\n            if const_expr(self.head_dim_padded < 96):\n                fp8_regs = _FP8_SMALL_HDIM_REGS[paged_kv_non_tma]\n                self.num_regs_softmax = fp8_regs[\"num_regs_softmax\"]\n                self.num_regs_correction = fp8_regs[\"num_regs_correction\"]\n                self.num_regs_other = fp8_regs[\"num_regs_other\"]\n            else:\n                fp8_tune = _FP8_TUNING_CONFIG.get(\n                    (\n                        self.use_2cta_instrs,\n                        self.is_causal,\n                        self.head_dim_padded,\n                        self.is_sm103,\n                    ),\n                    {},\n                )\n                if const_expr(\"ex2_emu_freq\" in fp8_tune):","sourceCodeStart":854,"sourceCodeEnd":890,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_sm100.py#L854-L890","documentation":"In block-scaled FP8 mode, the Q scale-factor tensor (SFQ) and K scale-factor tensor (SFK) must share dtype (e.g. both float8_e8m0fnu for MXFP8). Mismatched scale formats cannot be combined in the MMA.","triggerScenarios":"Invoking the kernel with qk_blockscaled=True and sfq.dtype != sfk.dtype, e.g. one e8m0 and the other float32.","commonSituations":"Custom quantizers emitting float32 scales for one side and e8m0 for the other; mixing MXFP8 and per-tensor scaled quantization between Q and K.","solutions":["Re-quantize so SFQ and SFK use the same scale dtype (typically torch.float8_e8m0fnu for MXFP8)","Check the quantization recipe produces symmetric scale formats for Q and K"],"exampleFix":"// before\nsfq_f32, sfk_e8m0 = quant(q), quant(k)\n// after\nsfq_e8m0, sfk_e8m0 = quant_mxfp8(q), quant_mxfp8(k)","handlingStrategy":"validation","validationCode":"if qk_blockscaled:\n    assert sfq.dtype == sfk.dtype, f'{sfq.dtype} != {sfk.dtype}'","typeGuard":"def scale_factors_consistent(sfq, sfk) -> bool: return sfq.dtype == sfk.dtype","tryCatchPattern":null,"preventionTips":["Use a single MXFP8 quantization helper for both Q and K so scale formats match","Check scale dtypes in unit tests for the quantizer"],"tags":["flash-attention","fp8","mxfp8","scale-factor","dtype-mismatch"],"backgroundTag":"dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}