sgl-project/sglang · error · NotImplementedError

Block sparsity + paged KV not supported on SM100

Error message

Block sparsity + paged KV not supported on SM100

What it means

The SM100 (Blackwell) FA4 kernel cannot combine block-sparse attention with paged KV cache (a page table). These two KV-access schemes are mutually exclusive in the current implementation.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_sm100.py:1552

                softmax_scale, self.score_mod
            )
        window_size_left = (
            Int32(window_size_left) if window_size_left is not None else None
        )
        window_size_right = (
            Int32(window_size_right) if window_size_right is not None else None
        )
        fastdiv_mods = utils.compute_fastdiv_mods(
            mQ, mK, self.qhead_per_kvhead, self.pack_gqa, aux_data.tensors, mPageTable
        )

        head_divmod = None
        if cutlass.const_expr(self.pack_gqa):
            head_divmod = FastDivmodDivisor(self.qhead_per_kvhead)

        self.use_block_sparsity = cutlass.const_expr(blocksparse_tensors is not None)
        if cutlass.const_expr(self.use_block_sparsity and mPageTable is not None):
            raise NotImplementedError(
                "Block sparsity + paged KV not supported on SM100"
            )
        if cutlass.const_expr(self.use_block_sparsity and self.is_varlen_q):
            assert const_expr(
                blocksparse_tensors.cu_total_m_blocks is not None
            ), "blocksparse_tensors.cu_total_m_blocks must be provided for varlen blocksparsity"

        # Launch the kernel synchronously
        self.kernel(
            mQ,
            mK,
            mV,
            mO,
            mLSE,
            mCuSeqlensQ,
            mCuSeqlensK,
            mSeqUsedQ,
            mSeqUsedK,

View on GitHub (pinned to 0132848349)

Solutions

1. Disable block sparsity when using paged KV
2. Or run with a non-paged (dense/varlen) KV layout for this layer/model
3. Track the upstream FA4 repo for blocksparse+paged support

Example fix

// before
attn(..., blocksparse_tensors=bs, page_table=pt)
// after
attn(..., page_table=pt)  # drop blocksparse_tensors

if blocksparse_tensors is not None and page_table is not None:
    raise ValueError('blocksparse + paged KV unsupported on SM100')  # fail early

def blocksparse_paged_supported(blocksparse_tensors, page_table, arch) -> bool:
    return not (blocksparse_tensors is not None and page_table is not None and arch >= 100)

try:
    attn(..., blocksparse_tensors=bs, page_table=pt)
except NotImplementedError:
    attn(...)  # fallback without blocksparse

When it happens

Trigger: Calling the SM100 kernel with both blocksparse_tensors and mPageTable (a KV cache page table) supplied, i.e. block-sparse attention on top of a paged KV cache.

Common situations: Serving with a paged KV cache (SGLang/vLLM-style) while enabling block-sparse attention (e.g. moe/blocksparse attention models) on B200/GB200.

Related errors

• Block sparsity + sheared bias is not supported on SM90
• Type mismatch: {self.q_dtype} != {self.k_dtype}
• Type mismatch: {self.q_dtype} != {self.v_dtype}
• The layout of mBias is wrong
• Custom user-provided score_mod is not supported on SM8x arch

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3c548a562ba2cb30. Report an issue: GitHub.