sgl-project/sglang · error · NotImplementedError

Block sparsity + paged KV not supported on SM100

Error message

Block sparsity + paged KV not supported on SM100

What it means

Unsupported combination on SM100 GPUs: block-sparse attention together with a paged KV cache. The SM100 kernel does not implement sparse block iteration over paged KV pages; use a non-paged layout or disable block sparsity.

Solutions

1. Disable block sparsity when using paged KV
2. Or run with a non-paged (dense/varlen) KV layout for this layer/model
3. Track the upstream FA4 repo for blocksparse+paged support

Example fix

// before
attn(..., blocksparse_tensors=bs, page_table=pt)
// after
attn(..., page_table=pt)  # drop blocksparse_tensors

if blocksparse_tensors is not None and page_table is not None:
    raise ValueError('blocksparse + paged KV unsupported on SM100')  # fail early

def blocksparse_paged_supported(blocksparse_tensors, page_table, arch) -> bool:
    return not (blocksparse_tensors is not None and page_table is not None and arch >= 100)

try:
    attn(..., blocksparse_tensors=bs, page_table=pt)
except NotImplementedError:
    attn(...)  # fallback without blocksparse

When it happens

Trigger: Calling the SM100 kernel with both blocksparse_tensors and mPageTable (a KV cache page table) supplied, i.e. block-sparse attention on top of a paged KV cache.

Common situations: Serving with a paged KV cache (SGLang/vLLM-style) while enabling block-sparse attention (e.g. moe/blocksparse attention models) on B200/GB200.

Related errors

• Block sparsity + sheared bias is not supported on SM90
• Custom user-provided score_mod is not supported on SM8x…
• hd256 forward non-varlen expects k rank 4 or 5, got rank
• hd256 forward varlen expects k rank 3 or 5, got rank
• The layout of k is not supported

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3c548a562ba2cb30. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/flash_fwd_sm100.py:1552

                softmax_scale, self.score_mod
            )
        window_size_left = (
            Int32(window_size_left) if window_size_left is not None else None
        )
        window_size_right = (
            Int32(window_size_right) if window_size_right is not None else None
        )
        fastdiv_mods = utils.compute_fastdiv_mods(
            mQ, mK, self.qhead_per_kvhead, self.pack_gqa, aux_data.tensors, mPageTable
        )

        head_divmod = None
        if cutlass.const_expr(self.pack_gqa):
            head_divmod = FastDivmodDivisor(self.qhead_per_kvhead)

        self.use_block_sparsity = cutlass.const_expr(blocksparse_tensors is not None)
        if cutlass.const_expr(self.use_block_sparsity and mPageTable is not None):
            raise NotImplementedError(
                "Block sparsity + paged KV not supported on SM100"
            )
        if cutlass.const_expr(self.use_block_sparsity and self.is_varlen_q):
            assert const_expr(
                blocksparse_tensors.cu_total_m_blocks is not None
            ), "blocksparse_tensors.cu_total_m_blocks must be provided for varlen blocksparsity"

        # Launch the kernel synchronously
        self.kernel(
            mQ,
            mK,
            mV,
            mO,
            mLSE,
            mCuSeqlensQ,
            mCuSeqlensK,
            mSeqUsedQ,
            mSeqUsedK,

View on GitHub (pinned to 0132848349)