sgl-project/sglang · error · ValueError

{name} must have dtype torch.int32

Error message

{name} must have dtype torch.int32

What it means

Raised when an auxiliary block-sparse metadata tensor (e.g. dq_write_order) does not have dtype torch.int32. The kernels index with 32-bit ints, so any other dtype is rejected.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:281

        expected_index_shape = (*expected_index_shape[:3], idx.shape[3])
    expanded_idx = _expand_sparsity_tensor(
        idx, expected_index_shape, f"{name}_block_idx", context, hint
    )
    return expanded_cnt, expanded_idx


def _check_and_expand_metadata_tensor(
    name: str,
    tensor: torch.Tensor | None,
    expected_shape: Tuple[int, ...],
    context: str | None,
    hint: str | Callable[[], str] | None,
    device: torch.device,
) -> torch.Tensor | None:
    if tensor is None:
        return None
    if tensor.dtype != torch.int32:
        raise ValueError(f"{name} must have dtype torch.int32")
    if tensor.device != device:
        raise ValueError(f"{name} must be on the same device as block sparse tensors")
    if not tensor.is_cuda:
        raise ValueError(f"{name} must live on CUDA")
    return _expand_sparsity_tensor(tensor, expected_shape, name, context, hint)


def get_block_sparse_expected_shapes(
    batch_size: int,
    num_head: int,
    seqlen_q: int,
    seqlen_k: int,
    m_block_size: int,
    n_block_size: int,
    q_stage: int,
) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int]]:
    """Return (expected_count_shape, expected_index_shape) for block sparse normalization."""
    m_block_size_effective = q_stage * m_block_size

View on GitHub (pinned to 0132848349)

Solutions

1. Cast to int32: dq_write_order = dq_write_order.to(torch.int32)
2. Create with explicit dtype: torch.arange(M, dtype=torch.int32, device='cuda')

Example fix

// before
order = torch.arange(num_m_blocks, device='cuda')  # int64
// after
order = torch.arange(num_m_blocks, dtype=torch.int32, device='cuda')

assert dq_write_order is None or dq_write_order.dtype == torch.int32

def is_valid_meta(t): return t is None or (t.dtype == torch.int32 and t.is_cuda)

When it happens

Trigger: Calling normalize_block_sparse_tensors with a dq_write_order tensor of dtype torch.long or torch.int16 instead of torch.int32.

Common situations: Creating dq_write_order via torch.arange(...) (defaults to int64) or torch.zeros without dtype=torch.int32, especially when enabling spt mode.

Related errors

• {name}_block tensors must have dtype torch.int32
• {name} must be on the same device as block sparse tensors
• {name} must live on CUDA
• cu_seqlens_q tensor must be Int32
• cu_seqlens_k tensor must be Int32

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/da43d5081ffe5a82. Report an issue: GitHub.