sgl-project/sglang · error · ValueError

Block sparse tensors

Error message

Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_block}.

What it means

Raised when the configured KV block size of the block-sparse mask does not match the attention kernel's BLOCK_SIZE_KV (n_block_size). FA4 block-sparse requires the sparse mask's KV granularity to equal the kernel tile size, otherwise indices would reference invalid KV tiles.

Solutions

1. Set the sparse mask's KV block size to the kernel's BLOCK_SIZE_KV (e.g. rebuild BlockMask with matching block size)
2. Pass sparse_block_size_kv equal to n_block_size when calling the op

Example fix

// before
BlockMask(block_size_kv=64, ...)
// after
BlockMask(block_size_kv=128, ...)  # == kernel BLOCK_SIZE_KV

assert sparse_block_size_kv in (None, n_block_size), f'must equal BLOCK_SIZE_KV={n_block_size}'

When it happens

Trigger: Calling infer_block_sparse_expected_shapes / normalize_block_sparse_config with sparse_block_size_kv != n_block_size (the kernel's KV tile, e.g. 128 vs a mask built with 64).

Common situations: Building a BlockMask with BLOCK_SIZE_KV=64 while the flash attention op is configured with n_block_size=128, or upgrading the kernel default tile size without regenerating the mask.

Related errors

• Attention backend override
• Block sparse tensors
• Block sparse tensors
• Invalid attention metadata values.Sparsity should be in
• _block_cnt and _block_idx must be on the same device

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0d90fe5556cddf5b. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:336

    sparse_block_size_q: int | None = None,
    sparse_block_size_kv: int | None = None,
) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int], int]:
    """Infer shapes and scaling for block-sparse tensors.

    Expectations:
    - mask_block_cnt is (B, H, M) and mask_block_idx is (B, H, M, N).
    - Batch/head dims may be 1 for broadcast, or match the requested sizes.
    - sparse_block_size_kv must match tile_n.
    - sparse_block_size_q must be a multiple of q_stage * tile_m.
    - If sparse_block_size_q is omitted and seqlen_q/num_m_blocks is ambiguous,
      the caller must provide block_size to disambiguate. TODO will make this required in a future PR.
    """
    base_m_block = q_stage * m_block_size
    base_n_block = n_block_size
    if sparse_block_size_kv is None:
        sparse_block_size_kv = base_n_block
    if sparse_block_size_kv != base_n_block:
        raise ValueError(
            f"Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_block}."
        )
    if tensors.mask_block_idx is None:
        raise ValueError(
            "mask_block_cnt and mask_block_idx must be provided for block sparsity."
        )
    num_m_blocks = tensors.mask_block_idx.shape[2]

    if sparse_block_size_q is None:
        sparse_block_size_q = get_sparse_q_block_size(tensors, seqlen_q)
        if sparse_block_size_q is None and base_m_block != 1:
            raise ValueError(
                f"Block sparse tensors{context} require explicit sparse_block_size[0] "
                f"to disambiguate block size for seqlen_q={seqlen_q} and num_m_blocks={num_m_blocks}."
            )
        if sparse_block_size_q is None:
            sparse_block_size_q = ceildiv(seqlen_q, num_m_blocks)

View on GitHub (pinned to 0132848349)