xai-org/x-algorithm · error · ValueError

Block sparsity expects sparse_block_size_q={subtile_factor *

Error message

Block sparsity expects sparse_block_size_q={subtile_factor * m_block_size} for subtile_factor={subtile_factor}.

What it means

In the backward normalize_block_sparse_config_bwd, the query-side sparse block size must equal subtile_factor * m_block_size — backward uses larger Q blocks due to subtiling. An explicit block_size[0] differing from that is rejected (None defaults correctly).

Source

Thrown at phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py:616

def normalize_block_sparse_config_bwd(
    tensors: BlockSparseTensorsTorch,
    *,
    batch_size: int,
    num_head: int,
    seqlen_q: int,
    seqlen_k: int,
    block_size: tuple[int, int],
    subtile_factor: int,
) -> tuple[BlockSparseTensorsTorch, Tuple[Tuple[bool, ...], ...] | None]:
    m_block_size, n_block_size = block_size
    if tensors.block_size is None:
        sparse_block_size_q, sparse_block_size_kv = subtile_factor * m_block_size, n_block_size
    else:
        sparse_block_size_q, sparse_block_size_kv = tensors.block_size
    if sparse_block_size_q != subtile_factor * m_block_size:
        raise ValueError(
            f"Block sparsity expects sparse_block_size_q={subtile_factor * m_block_size} "
            f"for subtile_factor={subtile_factor}."
        )
    if sparse_block_size_kv != n_block_size:
        raise ValueError(
            f"Block sparsity expects sparse_block_size[1]={n_block_size} to match tile_n."
        )
    expected_count_shape, expected_index_shape = get_block_sparse_expected_shapes_bwd(
        batch_size,
        num_head,
        seqlen_q,
        seqlen_k,
        m_block_size,
        n_block_size,
        subtile_factor,
    )
    normalized_tensors = normalize_block_sparse_tensors(
        tensors,

View on GitHub (pinned to 24c60942c5)

Solutions

1. Set block_size[0] = subtile_factor * tile_m for the backward pass, or use block_size=None so the default applies
2. Keep separate tensor configs for forward and backward if block sizes differ
3. Re-derive block size from the kernel's subtile_factor rather than hardcoding

Example fix

# before
tensors_bwd = tensors_fwd  # block_size=(64, 128), subtile_factor=2

# after
tensors_bwd = replace(tensors_fwd, block_size=(128, 128))  # subtile_factor*tile_m
# or block_size=None on both to use per-path defaults

expected_q = subtile_factor * m_block_size
assert tensors.block_size is None or tensors.block_size[0] == expected_q

When it happens

Trigger: Running the bwd normalization with tensors.block_size[0] set to tile_m (the forward value) instead of subtile_factor * tile_m; reusing the forward mask's block size for backward.

Common situations: Sharing one BlockSparseTensorsTorch between fwd and bwd with an explicit block_size; after a version change altered subtile_factor; exporting a mask with hardcoded block sizes.

Related errors

• Block sparsity expects sparse_block_size[1]={n_block_size} t
• Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_
• Block sparse tensors{context} m-block dimension {num_m_block
• Block sparsity requires sparse_block_size[1]={n_block_size}
• Varlen block sparsity requires sparse_block_size[0]={base_m_

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/4ea1427019198dd7. Report an issue: GitHub.