sgl-project/sglang · error · ValueError

Block sparse tensors{context} n-block dimension must be <= {

Error message

Block sparse tensors{context} n-block dimension must be <= {expected_n_blocks}.

What it means

Raised when mask_block_idx's n-block dimension exceeds expected_n_blocks = ceildiv(seqlen_k, sparse_block_size_kv). Larger is invalid; smaller is intentionally allowed (compact indices, since FA4 only reads 0..cnt-1 per tile).

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:394

        )
    for dim_name, cur, tgt in (
        ("batch", mask_block_cnt.shape[0], expected_count_shape[0]),
        ("head", mask_block_cnt.shape[1], expected_count_shape[1]),
    ):
        if cur != tgt and cur != 1:
            raise ValueError(
                f"Block sparse tensors{context} {dim_name} dim must be {tgt} or 1."
            )
    for dim_name, cur, tgt in (
        ("batch", mask_block_idx.shape[0], expected_index_shape[0]),
        ("head", mask_block_idx.shape[1], expected_index_shape[1]),
    ):
        if cur != tgt and cur != 1:
            raise ValueError(
                f"Block sparse tensors{context} {dim_name} dim must be {tgt} or 1."
            )
    if mask_block_cnt.shape[2] != mask_block_idx.shape[2]:
        raise ValueError(
            f"Block sparse tensors{context} must share the same m-block dimension."
        )
    # [Note] Allow Compact block sparse indices: FA4 only accesses indices 0..cnt-1
    # per query tile, so idx.shape[3] can be <= expected_n_blocks.
    if mask_block_idx.shape[3] > expected_n_blocks:
        raise ValueError(
            f"Block sparse tensors{context} n-block dimension must be <= {expected_n_blocks}."
        )
    if expected_m_blocks != num_m_blocks:
        raise ValueError(
            f"Block sparse tensors{context} m-block dimension {num_m_blocks} does not match "
            f"sparse_block_size_q={sparse_block_size_q}. "
            f"Set BlockSparseTensorsTorch.block_size to match the BlockMask BLOCK_SIZE."
        )
    return expected_count_shape, expected_index_shape, q_subtile_factor


def get_block_sparse_expected_shapes_bwd(

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate the mask for the current seqlen_k / KV cache length
2. Trim idx.shape[3] to expected_n_blocks (safe since only 0..cnt-1 are read): idx = idx[..., :expected_n_blocks]
3. Verify seqlen_k passed matches the actual k tensor length

Example fix

// before
idx = cached_idx  # (B,H,M,128) but only 64 kv blocks
// after
idx = cached_idx[..., :64]

expected_n = (seqlen_k + sparse_block_size_kv - 1) // sparse_block_size_kv
assert mask_block_idx.shape[3] <= expected_n

When it happens

Trigger: idx.shape[3]=128 when seqlen_k/block_size gives only 64 KV blocks — indices could point past the KV sequence.

Common situations: Mask generated for a longer KV length than the current call's k/v tensors (e.g. cached mask with larger max_seqlen), or wrong seqlen_k passed to the shape inference.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• Block sparse tensors{context} must have shapes (B, H, M) and
• Block sparse tensors{context} {dim_name} dim must be {tgt} o
• Block sparse tensors{context} must share the same m-block di
• Block sparse tensors{context} m-block dimension {num_m_block

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1ee4b710787cf4e3. Report an issue: GitHub.