sgl-project/sglang · error · ValueError

Block sparse tensors{context} must have shapes (B, H, M) and

Error message

Block sparse tensors{context} must have shapes (B, H, M) and (B, H, M, N).

What it means

Raised when the block-sparse count tensor is not 3-D or the index tensor is not 4-D. The kernel expects per (batch, head, m-block) counts and per (batch, head, m-block, n-block) indices.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:370

        if sparse_block_size_q is None:
            sparse_block_size_q = ceildiv(seqlen_q, num_m_blocks)

    if sparse_block_size_q % base_m_block != 0:
        raise ValueError(
            f"Block sparse tensors{context} have block size {sparse_block_size_q}, "
            f"which must be a multiple of {base_m_block}."
        )

    expected_m_blocks = ceildiv(seqlen_q, sparse_block_size_q)
    expected_n_blocks = ceildiv(seqlen_k, sparse_block_size_kv)
    q_subtile_factor = sparse_block_size_q // base_m_block
    expected_count_shape = (batch_size, num_head, expected_m_blocks)
    expected_index_shape = (batch_size, num_head, expected_m_blocks, expected_n_blocks)

    mask_block_cnt = tensors.mask_block_cnt
    mask_block_idx = tensors.mask_block_idx
    if mask_block_cnt is None or mask_block_idx is None:
        raise ValueError(
            "mask_block_cnt and mask_block_idx must be provided for block sparsity."
        )
    if mask_block_cnt.ndim != 3 or mask_block_idx.ndim != 4:
        raise ValueError(
            f"Block sparse tensors{context} must have shapes (B, H, M) and (B, H, M, N)."
        )
    for dim_name, cur, tgt in (
        ("batch", mask_block_cnt.shape[0], expected_count_shape[0]),
        ("head", mask_block_cnt.shape[1], expected_count_shape[1]),
    ):
        if cur != tgt and cur != 1:
            raise ValueError(
                f"Block sparse tensors{context} {dim_name} dim must be {tgt} or 1."
            )
    for dim_name, cur, tgt in (
        ("batch", mask_block_idx.shape[0], expected_index_shape[0]),
        ("head", mask_block_idx.shape[1], expected_index_shape[1]),
    ):

View on GitHub (pinned to 0132848349)

Solutions

1. Reshape/expand cnt to (B, H, M) — use .unsqueeze(1).expand(B, H, M) if head-uniform
2. Reshape idx to (B, H, M, N) with unsqueeze/expand as needed

Example fix

// before
cnt = torch.zeros((B, M), dtype=torch.int32, device='cuda')
// after
cnt = torch.zeros((B, 1, M), dtype=torch.int32, device='cuda').expand(B, H, M)

assert mask_block_cnt.ndim == 3 and mask_block_idx.ndim == 4

def valid_sparse_ranks(cnt, idx): return cnt.ndim == 3 and idx.ndim == 4

When it happens

Trigger: Passing mask_block_cnt with shape (B, M) (missing head dim) or mask_block_idx with shape (B, H, N) (missing m dim) to normalize_block_sparse_config.

Common situations: Migrating from a 2-D/3-D layout used by another block-sparse implementation or hand-building tensors without the head dimension.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• Block sparse tensors{context} {dim_name} dim must be {tgt} o
• Block sparse tensors{context} must share the same m-block di
• Block sparse tensors{context} n-block dimension must be <= {
• Block sparse tensors{context} m-block dimension {num_m_block

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/42188dc1b1f5c49e. Report an issue: GitHub.