sgl-project/sglang · error · TypeError

seqused_q tensor must be Int32

Error message

seqused_q tensor must be Int32

What it means

The optional seqused_q tensor (per-batch actual query sequence lengths, used to mask padding) must be Int32. Passing None is allowed when seqused masking is not used.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd.py:221

        # Get the data type and check if it is fp16 or bf16
        if const_expr(self.is_split_kv):
            # SplitKV writes float32 partial outputs; Q/K/V still fp16/bf16.
            if const_expr(not (mQ_type == mK_type == mV_type)):
                raise TypeError("Q/K/V must have the same data type")
            if const_expr(mO_type != Float32):
                raise TypeError("SplitKV partial output (mO) must be Float32")
        elif const_expr(not (mQ_type == mK_type == mV_type == mO_type)):
            raise TypeError("All tensors must have the same data type")
        if const_expr(mQ_type not in [cutlass.Float16, cutlass.BFloat16]):
            raise TypeError("Only Float16 or BFloat16 is supported")
        if const_expr(mLSE_type not in [None, Float32]):
            raise TypeError("LSE tensor must be Float32")
        if const_expr(mCuSeqlensQ_type not in [None, Int32]):
            raise TypeError("cu_seqlens_q tensor must be Int32")
        if const_expr(mCuSeqlensK_type not in [None, Int32]):
            raise TypeError("cu_seqlens_k tensor must be Int32")
        if const_expr(mSeqUsedQ_type not in [None, Int32]):
            raise TypeError("seqused_q tensor must be Int32")
        if const_expr(mSeqUsedK_type not in [None, Int32]):
            raise TypeError("seqused_k tensor must be Int32")
        assert mQ_type == self.dtype

    def _setup_attributes(self):
        # ///////////////////////////////////////////////////////////////////////////////
        # Shared memory layout: Q/K/V
        # ///////////////////////////////////////////////////////////////////////////////
        (
            sQ_layout_atom,
            sK_layout_atom,
            sV_layout_atom,
            sO_layout_atom,
            sP_layout_atom,
        ) = self._get_smem_layout_atom()
        self.sQ_layout = cute.tile_to_shape(
            sQ_layout_atom,
            (self.tile_m, self.tile_hdim),

View on GitHub (pinned to 0132848349)

Solutions

1. Cast: seqused_q = seqused_q.to(torch.int32)
2. Allocate length tables with dtype=torch.int32 from the start
3. Also cast seqused_k (checked next)

Example fix

// before
seqused_q = seq_lens  # int64 from dataloader
// after
seqused_q = seq_lens.to(torch.int32)

if seqused_q is not None:
    assert seqused_q.dtype == torch.int32

def int32_or_none(t) -> bool:
    return t is None or t.dtype == torch.int32

When it happens

Trigger: Providing seqused_q with dtype Int64 or anything other than Int32 to FlashAttentionForward.

Common situations: Passing seq_lens tensors straight from a dataloader (commonly int64); enabling page/padding masking with tensors prepared for a different backend that accepts int64.

Related errors

• seqused_k tensor must be Int32
• cu_seqlens_q tensor must be Int32
• cu_seqlens_k tensor must be Int32
• SplitKV partial output (mO) must be Float32
• All tensors must have the same data type

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/597daef9eeadd2f8. Report an issue: GitHub.