xai-org/x-algorithm · error · ValueError

{name}_block tensors must have dtype torch.int32

Error message

{name}_block tensors must have dtype torch.int32

What it means

Block-sparsity cnt/idx tensors must be torch.int32 because the downstream CUDA kernels consume 32-bit indices. _check_and_expand_block rejects any other dtype before expansion/validation, since int64 metadata would be reinterpreted incorrectly or silently truncated.

Source

Thrown at phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py:240

def _check_and_expand_block(
    name: str,
    cnt: torch.Tensor | None,
    idx: torch.Tensor | None,
    expected_count_shape: Tuple[int, ...],
    expected_index_shape: Tuple[int, ...],
    context: str | None,
    hint: str | Callable[[], str] | None,
) -> Tuple[torch.Tensor | None, torch.Tensor | None]:
    if (cnt is None) != (idx is None):
        raise ValueError(
            f"{name}_block_cnt and {name}_block_idx must both be provided or both be None"
        )
    if cnt is None or idx is None:
        return None, None
    if cnt.dtype != torch.int32 or idx.dtype != torch.int32:
        raise ValueError(f"{name}_block tensors must have dtype torch.int32")
    if cnt.device != idx.device:
        raise ValueError(f"{name}_block_cnt and {name}_block_idx must be on the same device")
    if not cnt.is_cuda or not idx.is_cuda:
        raise ValueError(f"{name}_block tensors must live on CUDA")
    expanded_cnt = _expand_sparsity_tensor(
        cnt, expected_count_shape, f"{name}_block_cnt", context, hint
    )
    if idx.ndim == 4 and idx.shape[3] <= expected_index_shape[3]:
        expected_index_shape = (*expected_index_shape[:3], idx.shape[3])
    expanded_idx = _expand_sparsity_tensor(
        idx, expected_index_shape, f"{name}_block_idx", context, hint
    )
    return expanded_cnt, expanded_idx


def _check_and_expand_metadata_tensor(
    name: str,
    tensor: torch.Tensor | None,

View on GitHub (pinned to 24c60942c5)

Solutions

1. Create/convert with dtype=torch.int32: torch.tensor(..., dtype=torch.int32) or t.int()
2. When saving checkpoints, keep the dtype or cast on load: cnt.load(...).to(torch.int32)
3. Add a unit test asserting metadata dtypes before the kernel call

Example fix

# before
cnt = torch.zeros((B, H, M), device='cuda')  # int64 by default
# after
cnt = torch.zeros((B, H, M), dtype=torch.int32, device='cuda')

cnt = None if cnt is None else cnt.to(torch.int32)
idx = None if idx is None else idx.to(torch.int32)

def is_int32_cuda_pair(cnt, idx) -> bool:
    return (
        (cnt is None) == (idx is None)
        and (cnt is None or (cnt.dtype == torch.int32 and cnt.is_cuda))
        and (idx is None or (idx.dtype == torch.int32 and idx.is_cuda))
    )

When it happens

Trigger: Passing {name}_block_cnt or {name}_block_idx created with torch.zeros/ones defaults (int64), or loaded from a numpy array (default int64 on Linux), into normalize_block_sparse_tensors.

Common situations: torch tensor factory defaults producing int64; saving/loading metadata through numpy which upgrades to int64; exporting from a different framework without an explicit dtype.

Related errors

• {tensor_name}{context_clause} with shape {tensor.shape} cann
• {name}_block_cnt and {name}_block_idx must both be provided
• The value of top_feedforward specified ({top_feedforward}) d
• Only int32 is supported for unique.
• block-sparse arrays cover {bs_num_blocks} m-tiles but the ke

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/f5f033cc6d5b4488. Report an issue: GitHub.