xai-org/x-algorithm · error · ValueError

Block sparsity expects sparse_block_size[1]={n_block_size} t

Error message

Block sparsity expects sparse_block_size[1]={n_block_size} to match tile_n.

What it means

Backward counterpart of error 353: sparse_block_size_kv (block_size[1]) must match tile_n in normalize_block_sparse_config_bwd. Fires when an explicit KV block size differs from the kernel's N tile.

Source

Thrown at phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py:621

    batch_size: int,
    num_head: int,
    seqlen_q: int,
    seqlen_k: int,
    block_size: tuple[int, int],
    subtile_factor: int,
) -> tuple[BlockSparseTensorsTorch, Tuple[Tuple[bool, ...], ...] | None]:
    m_block_size, n_block_size = block_size
    if tensors.block_size is None:
        sparse_block_size_q, sparse_block_size_kv = subtile_factor * m_block_size, n_block_size
    else:
        sparse_block_size_q, sparse_block_size_kv = tensors.block_size
    if sparse_block_size_q != subtile_factor * m_block_size:
        raise ValueError(
            f"Block sparsity expects sparse_block_size_q={subtile_factor * m_block_size} "
            f"for subtile_factor={subtile_factor}."
        )
    if sparse_block_size_kv != n_block_size:
        raise ValueError(
            f"Block sparsity expects sparse_block_size[1]={n_block_size} to match tile_n."
        )
    expected_count_shape, expected_index_shape = get_block_sparse_expected_shapes_bwd(
        batch_size,
        num_head,
        seqlen_q,
        seqlen_k,
        m_block_size,
        n_block_size,
        subtile_factor,
    )
    normalized_tensors = normalize_block_sparse_tensors(
        tensors,
        expected_count_shape=expected_count_shape,
        expected_index_shape=expected_index_shape,
        context="_flash_attn_bwd",
        hint=lambda: (
            f"Backward expects Q-direction block-sparse tensors (q_mask_cnt/q_mask_idx, "

View on GitHub (pinned to 24c60942c5)

Solutions

1. Set block_size[1] to tile_n or use block_size=None
2. Rebuild masks with the backward kernel's tile_n
3. Align mask generator and kernel tile config in one place

Example fix

# before
tensors_bwd = BlockSparseTensorsTorch(..., block_size=(128, 256))  # tile_n=128

# after
tensors_bwd = BlockSparseTensorsTorch(..., block_size=(128, 128))
# or block_size=None

assert tensors.block_size is None or tensors.block_size[1] == tile_n

When it happens

Trigger: bwd normalization with tensors.block_size = (q, kv) where kv != tile_n; a shared fwd/bwd tensor config whose kv block size was chosen for a different tile config.

Common situations: Reusing BlockMask BLOCK_SIZE across kernel configs; tile_n changed by a kernel version bump while masks stayed fixed.

Related errors

• Block sparsity requires sparse_block_size[1]={n_block_size}
• Block sparsity expects sparse_block_size_q={subtile_factor *
• Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_
• Block sparse tensors{context} m-block dimension {num_m_block
• Varlen block sparsity requires sparse_block_size[0]={base_m_

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/c853243f5ae1e081. Report an issue: GitHub.