{"record":{"id":"7a9c71e6b671abc1","repo":"sgl-project/sglang","slug":"lse-tensor-must-be-float32","errorCode":null,"errorMessage":"LSE tensor must be Float32","messagePattern":"LSE tensor must be Float32","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd.py","lineNumber":215,"sourceCode":"        mLSE_type: Type[cutlass.Numeric] | None,\n        mCuSeqlensQ_type: Type[cutlass.Numeric] | None,\n        mCuSeqlensK_type: Type[cutlass.Numeric] | None,\n        mSeqUsedQ_type: Type[cutlass.Numeric] | None,\n        mSeqUsedK_type: Type[cutlass.Numeric] | None,\n    ):\n        # Get the data type and check if it is fp16 or bf16\n        if const_expr(self.is_split_kv):\n            # SplitKV writes float32 partial outputs; Q/K/V still fp16/bf16.\n            if const_expr(not (mQ_type == mK_type == mV_type)):\n                raise TypeError(\"Q/K/V must have the same data type\")\n            if const_expr(mO_type != Float32):\n                raise TypeError(\"SplitKV partial output (mO) must be Float32\")\n        elif const_expr(not (mQ_type == mK_type == mV_type == mO_type)):\n            raise TypeError(\"All tensors must have the same data type\")\n        if const_expr(mQ_type not in [cutlass.Float16, cutlass.BFloat16]):\n            raise TypeError(\"Only Float16 or BFloat16 is supported\")\n        if const_expr(mLSE_type not in [None, Float32]):\n            raise TypeError(\"LSE tensor must be Float32\")\n        if const_expr(mCuSeqlensQ_type not in [None, Int32]):\n            raise TypeError(\"cu_seqlens_q tensor must be Int32\")\n        if const_expr(mCuSeqlensK_type not in [None, Int32]):\n            raise TypeError(\"cu_seqlens_k tensor must be Int32\")\n        if const_expr(mSeqUsedQ_type not in [None, Int32]):\n            raise TypeError(\"seqused_q tensor must be Int32\")\n        if const_expr(mSeqUsedK_type not in [None, Int32]):\n            raise TypeError(\"seqused_k tensor must be Int32\")\n        assert mQ_type == self.dtype\n\n    def _setup_attributes(self):\n        # ///////////////////////////////////////////////////////////////////////////////\n        # Shared memory layout: Q/K/V\n        # ///////////////////////////////////////////////////////////////////////////////\n        (\n            sQ_layout_atom,\n            sK_layout_atom,\n            sV_layout_atom,","sourceCodeStart":197,"sourceCodeEnd":233,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd.py#L197-L233","documentation":"The optional LSE (log-sum-exp) output tensor of the flash-attention forward op must be Float32 when provided. LSE stores per-head log-normalization constants in fp32 for numerical stability and for later combine/reduction steps.","triggerScenarios":"Calling FlashAttentionForward with an mLSE tensor whose element type is Float16/BFloat16 instead of Float32 (passing None is allowed and skips the check).","commonSituations":"Allocating LSE with the same dtype as O for symmetry; reusing an fp16 workspace buffer for LSE; LSE needed downstream by a split-KV combine or backwards pass that assumes fp32.","solutions":["Allocate LSE as float32: torch.empty((b,h,s), dtype=torch.float32, device=...)","Pass None if you don't need LSE output"],"exampleFix":"// before\nLSE = torch.empty((b,h,s), dtype=torch.float16, device='cuda')\n// after\nLSE = torch.empty((b,h,s), dtype=torch.float32, device='cuda')","handlingStrategy":"type-guard","validationCode":"if LSE is not None:\n    assert LSE.dtype == torch.float32","typeGuard":"def valid_lse(LSE) -> bool:\n    return LSE is None or LSE.dtype == torch.float32","tryCatchPattern":null,"preventionTips":["Allocate LSE explicitly with dtype=torch.float32","Pass None when LSE is not needed"],"tags":["cuda","dtype","flash-attention","cutlass","lse"],"backgroundTag":null,"analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}