sgl-project/sglang · error · ValueError

Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_

Error message

Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_block}.

What it means

Raised when the configured KV block size of the block-sparse mask does not match the attention kernel's BLOCK_SIZE_KV (n_block_size). FA4 block-sparse requires the sparse mask's KV granularity to equal the kernel tile size, otherwise indices would reference invalid KV tiles.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:336

    sparse_block_size_q: int | None = None,
    sparse_block_size_kv: int | None = None,
) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int], int]:
    """Infer shapes and scaling for block-sparse tensors.

    Expectations:
    - mask_block_cnt is (B, H, M) and mask_block_idx is (B, H, M, N).
    - Batch/head dims may be 1 for broadcast, or match the requested sizes.
    - sparse_block_size_kv must match tile_n.
    - sparse_block_size_q must be a multiple of q_stage * tile_m.
    - If sparse_block_size_q is omitted and seqlen_q/num_m_blocks is ambiguous,
      the caller must provide block_size to disambiguate. TODO will make this required in a future PR.
    """
    base_m_block = q_stage * m_block_size
    base_n_block = n_block_size
    if sparse_block_size_kv is None:
        sparse_block_size_kv = base_n_block
    if sparse_block_size_kv != base_n_block:
        raise ValueError(
            f"Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_block}."
        )
    if tensors.mask_block_idx is None:
        raise ValueError(
            "mask_block_cnt and mask_block_idx must be provided for block sparsity."
        )
    num_m_blocks = tensors.mask_block_idx.shape[2]

    if sparse_block_size_q is None:
        sparse_block_size_q = get_sparse_q_block_size(tensors, seqlen_q)
        if sparse_block_size_q is None and base_m_block != 1:
            raise ValueError(
                f"Block sparse tensors{context} require explicit sparse_block_size[0] "
                f"to disambiguate block size for seqlen_q={seqlen_q} and num_m_blocks={num_m_blocks}."
            )
        if sparse_block_size_q is None:
            sparse_block_size_q = ceildiv(seqlen_q, num_m_blocks)

View on GitHub (pinned to 0132848349)

Solutions

1. Set the sparse mask's KV block size to the kernel's BLOCK_SIZE_KV (e.g. rebuild BlockMask with matching block size)
2. Pass sparse_block_size_kv equal to n_block_size when calling the op

Example fix

// before
BlockMask(block_size_kv=64, ...)
// after
BlockMask(block_size_kv=128, ...)  # == kernel BLOCK_SIZE_KV

assert sparse_block_size_kv in (None, n_block_size), f'must equal BLOCK_SIZE_KV={n_block_size}'

When it happens

Trigger: Calling infer_block_sparse_expected_shapes / normalize_block_sparse_config with sparse_block_size_kv != n_block_size (the kernel's KV tile, e.g. 128 vs a mask built with 64).

Common situations: Building a BlockMask with BLOCK_SIZE_KV=64 while the flash attention op is configured with n_block_size=128, or upgrading the kernel default tile size without regenerating the mask.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• {name}_block_cnt and {name}_block_idx must both be provided
• {name}_block tensors must have dtype torch.int32
• {name}_block_cnt and {name}_block_idx must be on the same de
• {name}_block tensors must live on CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0d90fe5556cddf5b. Report an issue: GitHub.