{"record":{"id":"946bf2e8643f0e1d","repo":"sgl-project/sglang","slug":"type-must-match-self-a-dtype-self-b-dtype","errorCode":null,"errorMessage":"Type must match: {self.a_dtype} != {self.b_dtype}","messagePattern":"Type must match: (.+?) != (.+?)","errorType":"exception","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py","lineNumber":492,"sourceCode":"        :type stream: cuda.CUstream\n        :param epilogue_op: Optional elementwise lambda function to apply to the output tensor\n        :type epilogue_op: cutlass.Constexpr\n        :param use_pdl: Enable Programmatic Dependent Launch.\n        :type use_pdl: cutlass.Constexpr\n        :raises TypeError: If input data types are incompatible with the MMA instruction.\n        \"\"\"\n        # Setup static attributes before smem/grid/tma computation\n        self.a_dtype: Type[cutlass.Numeric] = a_tensor.element_type\n        self.b_dtype: Type[cutlass.Numeric] = b_tensor.element_type\n        self.sf_dtype: Type[cutlass.Numeric] = sfa_tensor.element_type\n        self.c_dtype: Type[cutlass.Numeric] = c_tensor.element_type\n        self.a_major_mode = utils.LayoutEnum.from_tensor(a_tensor).mma_major_mode()\n        self.b_major_mode = utils.LayoutEnum.from_tensor(b_tensor).mma_major_mode()\n        self.c_layout = utils.LayoutEnum.from_tensor(c_tensor)\n\n        # Check if input data types are compatible with MMA instruction\n        if cutlass.const_expr(self.a_dtype != self.b_dtype):\n            raise TypeError(f\"Type must match: {self.a_dtype} != {self.b_dtype}\")\n\n        # Setup attributes that dependent on gemm inputs\n        self._setup_attributes()\n\n        # Setup sfa/sfb tensor by filling A/B tensor to scale factor atom layout\n        # ((Atom_M, Rest_M),(Atom_K, Rest_K),RestL)\n        sfa_layout = blockscaled_utils.tile_atom_to_shape_SF(\n            a_tensor.shape, self.sf_vec_size\n        )\n        sfa_tensor = cute.make_tensor(sfa_tensor.iterator, sfa_layout)\n\n        # ((Atom_N, Rest_N),(Atom_K, Rest_K),RestL)\n        sfb_layout = blockscaled_utils.tile_atom_to_shape_SF(\n            b_tensor.shape, self.sf_vec_size\n        )\n        sfb_tensor = cute.make_tensor(sfb_tensor.iterator, sfb_layout)\n\n        # Determine if we need to generate scale factor C for quantization","sourceCodeStart":474,"sourceCodeEnd":510,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/quantization/nvfp4_gemm_swiglu_nvfp4_quant.py#L474-L510","documentation":"Raised inside the CUTLASS kernel's __call__ when the A (activation) tensor and B (weight) tensor of the fused NVFP4 GEMM+SwiGLU+quant operation have different element dtypes. The MMA instruction requires both operands to be the same type; NVFP4 paths expect float4_e2m1fn on both sides. The check runs at kernel setup time before any tile is computed.","triggerScenarios":"Calling the fused nvfp4 gemm/swiglu op with an activation tensor quantized to one dtype (e.g. float4_e2m1fn) and a weight tensor in another (e.g. uint8-packed FP4 without decode, or fp8/bf16), or passing a dequantized B while A stays FP4.","commonSituations":"Mixing quantization schemes between layers (e.g. w4a16 weights fed to an w4a4 kernel), incorrect weight preprocessing in process_weights_after_loading that leaves B in a different dtype, or version changes where the packed-FP4 representation changed.","solutions":["Verify both a and b are torch.float4_e2m1fn before calling the kernel","Check the weight loader / process_weights_after_loading path that repacks B into FP4","Ensure the same quant config (NVFP4) is applied to both activations and weights, not just one side"],"exampleFix":"// before\nout = fused_op(a_fp8, b_fp4)\n// after\nassert a.dtype == torch.float4_e2m1fn and b.dtype == torch.float4_e2m1fn\nout = fused_op(a, b)","handlingStrategy":"validation","validationCode":"assert a.dtype == b.dtype == torch.float4_e2m1fn, (a.dtype, b.dtype)","typeGuard":"def is_nvfp4_pair(a, b): return a.dtype == b.dtype == torch.float4_e2m1fn","tryCatchPattern":null,"preventionTips":["Quantize activations and weights with the same NVFP4 config","Add dtype asserts in weight-preprocessing unit tests"],"tags":["nvfp4","quantization","dtype-mismatch","cutlass","gpu-kernel"],"backgroundTag":"dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}