sgl-project/sglang · error · TypeError

cu_seqlens_k tensor must be Int32

Error message

cu_seqlens_k tensor must be Int32

What it means

The optional cu_seqlens_k tensor (cumulative sequence lengths for varlen batched keys/values) must be Int32, mirroring the check on cu_seqlens_q. Passing None is allowed outside varlen mode.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd.py:219

        mSeqUsedK_type: Type[cutlass.Numeric] | None,
    ):
        # Get the data type and check if it is fp16 or bf16
        if const_expr(self.is_split_kv):
            # SplitKV writes float32 partial outputs; Q/K/V still fp16/bf16.
            if const_expr(not (mQ_type == mK_type == mV_type)):
                raise TypeError("Q/K/V must have the same data type")
            if const_expr(mO_type != Float32):
                raise TypeError("SplitKV partial output (mO) must be Float32")
        elif const_expr(not (mQ_type == mK_type == mV_type == mO_type)):
            raise TypeError("All tensors must have the same data type")
        if const_expr(mQ_type not in [cutlass.Float16, cutlass.BFloat16]):
            raise TypeError("Only Float16 or BFloat16 is supported")
        if const_expr(mLSE_type not in [None, Float32]):
            raise TypeError("LSE tensor must be Float32")
        if const_expr(mCuSeqlensQ_type not in [None, Int32]):
            raise TypeError("cu_seqlens_q tensor must be Int32")
        if const_expr(mCuSeqlensK_type not in [None, Int32]):
            raise TypeError("cu_seqlens_k tensor must be Int32")
        if const_expr(mSeqUsedQ_type not in [None, Int32]):
            raise TypeError("seqused_q tensor must be Int32")
        if const_expr(mSeqUsedK_type not in [None, Int32]):
            raise TypeError("seqused_k tensor must be Int32")
        assert mQ_type == self.dtype

    def _setup_attributes(self):
        # ///////////////////////////////////////////////////////////////////////////////
        # Shared memory layout: Q/K/V
        # ///////////////////////////////////////////////////////////////////////////////
        (
            sQ_layout_atom,
            sK_layout_atom,
            sV_layout_atom,
            sO_layout_atom,
            sP_layout_atom,
        ) = self._get_smem_layout_atom()
        self.sQ_layout = cute.tile_to_shape(

View on GitHub (pinned to 0132848349)

Solutions

1. Cast to int32: cu_seqlens_k = cu_seqlens_k.to(torch.int32)
2. Apply the same cast to cu_seqlens_q to satisfy its parallel check

Example fix

// before
cu_seqlens_k = torch.cumsum(kv_lens, 0)  # int64
// after
cu_seqlens_k = torch.cumsum(kv_lens, 0).to(torch.int32)

if cu_seqlens_k is not None:
    assert cu_seqlens_k.dtype == torch.int32

def int32_or_none(t) -> bool:
    return t is None or t.dtype == torch.int32

When it happens

Trigger: Supplying cu_seqlens_k as Int64 or another non-Int32 integer dtype to FlashAttentionForward in varlen mode.

Common situations: Deriving cu_seqlens_k via torch.cumsum or slicing a precomputed int64 table; mismatched casting where q version was cast but k version was forgotten.

Related errors

• cu_seqlens_q tensor must be Int32
• seqused_q tensor must be Int32
• seqused_k tensor must be Int32
• SplitKV partial output (mO) must be Float32
• All tensors must have the same data type

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b5f604eac35547f6. Report an issue: GitHub.