sgl-project/sglang · error · ValueError

Block sparse tensors{context} {dim_name} dim must be {tgt} o

Error message

Block sparse tensors{context} {dim_name} dim must be {tgt} or 1.

What it means

Raised when mask_block_cnt's batch or head dimension is neither the expected value nor 1 (broadcastable). Dimensions of size 1 are expanded; any other mismatch is rejected.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:374

        raise ValueError(
            f"Block sparse tensors{context} have block size {sparse_block_size_q}, "
            f"which must be a multiple of {base_m_block}."
        )

    expected_m_blocks = ceildiv(seqlen_q, sparse_block_size_q)
    expected_n_blocks = ceildiv(seqlen_k, sparse_block_size_kv)
    q_subtile_factor = sparse_block_size_q // base_m_block
    expected_count_shape = (batch_size, num_head, expected_m_blocks)
    expected_index_shape = (batch_size, num_head, expected_m_blocks, expected_n_blocks)

    mask_block_cnt = tensors.mask_block_cnt
    mask_block_idx = tensors.mask_block_idx
    if mask_block_cnt is None or mask_block_idx is None:
        raise ValueError(
            "mask_block_cnt and mask_block_idx must be provided for block sparsity."
        )
    if mask_block_cnt.ndim != 3 or mask_block_idx.ndim != 4:
        raise ValueError(
            f"Block sparse tensors{context} must have shapes (B, H, M) and (B, H, M, N)."
        )
    for dim_name, cur, tgt in (
        ("batch", mask_block_cnt.shape[0], expected_count_shape[0]),
        ("head", mask_block_cnt.shape[1], expected_count_shape[1]),
    ):
        if cur != tgt and cur != 1:
            raise ValueError(
                f"Block sparse tensors{context} {dim_name} dim must be {tgt} or 1."
            )
    for dim_name, cur, tgt in (
        ("batch", mask_block_idx.shape[0], expected_index_shape[0]),
        ("head", mask_block_idx.shape[1], expected_index_shape[1]),
    ):
        if cur != tgt and cur != 1:
            raise ValueError(
                f"Block sparse tensors{context} {dim_name} dim must be {tgt} or 1."
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Set the head dim to num_head or squeeze to 1 for a head-shared mask: cnt = cnt[:, :1] / build with H=1
2. Regenerate the mask for the current model's head count

Example fix

// before
cnt = torch.zeros((B, 4, M), ...)  # num_head=8
// after
cnt = torch.zeros((B, 1, M), ...)  # broadcast over heads

for name, cur, tgt in (('b', cnt.shape[0], B), ('h', cnt.shape[1], H)):
    assert cur in (tgt, 1), f'{name} dim {cur} must be {tgt} or 1'

When it happens

Trigger: mask_block_cnt.shape[1]=4 while the attention call has num_head=8 (and not 1), during normalize_block_sparse_config.

Common situations: Model change (different num_heads) reusing an old mask, or GQA conversion that forgot to align the head dim of the count tensor.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• Block sparse tensors{context} must have shapes (B, H, M) and
• Block sparse tensors{context} must share the same m-block di
• Block sparse tensors{context} n-block dimension must be <= {
• Block sparse tensors{context} m-block dimension {num_m_block

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/57cd518c18be2d8a. Report an issue: GitHub.