{"record":{"id":"e657ab8fafb794a7","repo":"xai-org/x-algorithm","slug":"type-mismatch-self-q-dtype-self-v-dtype","errorCode":null,"errorMessage":"Type mismatch: {self.q_dtype} != {self.v_dtype}","messagePattern":"Type mismatch: (.+?) != (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/cutedsl/ranker_fa4/flash_fwd_sm100.py","lineNumber":492,"sourceCode":"            LSE_layout_transpose = [3, 2, 1, 0] if const_expr(mCuSeqlensQ is None) else [2, 1, 0]\n            num_splits = mO.shape[0]\n        else:\n            O_layout_transpose = [1, 3, 2, 0] if const_expr(mCuSeqlensQ is None) else [0, 2, 1]\n            LSE_layout_transpose = [2, 1, 0] if const_expr(mCuSeqlensQ is None) else [1, 0]\n            num_splits = Int32(1)\n        mO = cute.make_tensor(mO.iterator, cute.select(mO.layout, mode=O_layout_transpose))\n        mLSE = (\n            cute.make_tensor(mLSE.iterator, cute.select(mLSE.layout, mode=LSE_layout_transpose))\n            if const_expr(mLSE is not None)\n            else None\n        )\n        V_layout_transpose = [1, 0, 2, 3] if const_expr(mCuSeqlensK is None) else [1, 0, 2]\n        mV = cute.make_tensor(mV.iterator, cute.select(mV.layout, mode=V_layout_transpose))\n\n        if const_expr(self.q_dtype != self.k_dtype):\n            raise TypeError(f\"Type mismatch: {self.q_dtype} != {self.k_dtype}\")\n        if const_expr(self.q_dtype != self.v_dtype):\n            raise TypeError(f\"Type mismatch: {self.q_dtype} != {self.v_dtype}\")\n        if const_expr(self.q_dtype.width == 8):\n            paged_kv_non_tma = not self.use_tma_KV\n            if const_expr(self.head_dim_padded < 96):\n                fp8_regs = _FP8_SMALL_HDIM_REGS[paged_kv_non_tma]\n                self.num_regs_softmax = fp8_regs[\"num_regs_softmax\"]\n                self.num_regs_correction = fp8_regs[\"num_regs_correction\"]\n                self.num_regs_other = fp8_regs[\"num_regs_other\"]\n            else:\n                fp8_tune = _FP8_TUNING_CONFIG.get(\n                    (self.use_2cta_instrs, self.is_causal, self.head_dim_padded, self.is_sm103), {}\n                )\n                if const_expr(\"ex2_emu_freq\" in fp8_tune):\n                    self._tune = {**self._tune, **fp8_tune}\n                    self.enable_ex2_emu = self._tune[\"ex2_emu_freq\"] > 0\n                if const_expr(not paged_kv_non_tma and \"num_regs_softmax\" in fp8_tune):\n                    self.num_regs_softmax = fp8_tune[\"num_regs_softmax\"]\n                    self.num_regs_correction = fp8_tune[\"num_regs_correction\"]\n                    self.num_regs_other = 512 - self.num_regs_softmax * 2 - self.num_regs_correction","sourceCodeStart":474,"sourceCodeEnd":510,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/cutedsl/ranker_fa4/flash_fwd_sm100.py#L474-L510","documentation":"The SM100 flash-attention forward kernel requires Q, V (and K) to have identical element types since the MMA descriptors and TMA copies are specialized per dtype. __call__ checks self.q_dtype against self.v_dtype and raises TypeError when they differ.","triggerScenarios":"Calling the forward kernel with Q and V in different dtypes (e.g. Q bf16, V fp8e4m3), triggering the const_expr(q_dtype != v_dtype) branch.","commonSituations":"Enabling an fp8 V cache while keeping Q in bf16, mixed-precision inference configs that quantize only V, or a projection layer output dtype differing from the Q path.","solutions":["Cast Q to match V's dtype (or vice versa) before the call: q = q.to(v.dtype)","Unify all three tensors to one dtype in the model/config","If fp8 is intended, quantize Q, K, and V together with the same fp8 variant","Add an upfront assert q.dtype == v.dtype in the caller to fail before kernel launch"],"exampleFix":"// before\nout = flash_attn(q, k, v)  # q: bf16, v: fp8\n// after\nq = q.to(v.dtype); k = k.to(v.dtype)\nout = flash_attn(q, k, v)","handlingStrategy":"type-guard","validationCode":"assert q.dtype == v.dtype == k.dtype, f'{q.dtype}, {k.dtype}, {v.dtype}'","typeGuard":"def same_dtype(*ts):\n    d = ts[0].dtype\n    return all(t.dtype == d for t in ts)","tryCatchPattern":null,"preventionTips":["Centralize dtype policy in one config value used for Q, K, and V","When enabling quantization, quantize the whole KV/Q set together, never just one tensor"],"tags":["dtype","flash-attention","type-mismatch","cuda"],"backgroundTag":"dtype-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}