sgl-project/sglang · error · ValueError

indices must be int32, got {indices.dtype}

Error message

indices must be int32, got {indices.dtype}

What it means

The kernel reads index values as 32-bit integers; indices must be torch.int32. int64 (PyTorch's default for topk/arange outputs) or int16 indices are rejected because the kernel would reinterpret them with the wrong width.

Source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:370

            f"multiple of 64, got {h_q}"
        )

    if h_kv != 1:
        raise ValueError(f"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}")

    if d_qk not in (512, 576):
        raise ValueError(
            f"sparse_mla_q8kv8_prefill_fwd supports d_qk=512/576, got {d_qk}"
        )

    if indices.shape[:2] != (s_q, h_kv):
        raise ValueError(
            "indices must have shape "
            f"({s_q}, {h_kv}, topk), got {tuple(indices.shape)}"
        )

    if indices.dtype != torch.int32:
        raise ValueError(f"indices must be int32, got {indices.dtype}")

    if topk == 0 or topk % 128 != 0:
        raise ValueError(
            "Q8KV8 sparse-prefill topk width must be a positive multiple of 128, "
            f"got {topk}"
        )

    if topk_length is not None:
        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:
            raise ValueError(
                f"topk_length must be int32 with shape ({s_q},), got "
                f"{tuple(topk_length.shape)}/{topk_length.dtype}"
            )
        if not topk_length.is_cuda:
            raise ValueError("topk_length must be a CUDA tensor")
        if topk_length.device != device:
            raise ValueError(
                "topk_length must be on q's device "

View on GitHub (pinned to 0132848349)

Solutions

1. Cast before the call: indices = indices.to(torch.int32)
2. Check any arithmetic done on indices (clamping, adding offsets) preserves int32 dtype
3. Verify the index-producing kernel/triton op writes int32 output

Example fix

// before
indices = torch.topk(scores, k=topk, dim=-1).indices  # int64
// after
indices = torch.topk(scores, k=topk, dim=-1).indices.to(torch.int32)

if indices.dtype != torch.int32: indices = indices.to(torch.int32)

def is_int32(t: torch.Tensor) -> bool:
    return t.dtype == torch.int32

When it happens

Trigger: Passing torch.topk(...).indices (int64) directly without casting to int32.

Common situations: Default PyTorch integer ops producing int64; porting from a backend that accepted int64; producer kernels emitting int32 but intermediate torch ops promoting to int64.

Related errors

• q must be torch.float8_e4m3fn, got {q.dtype}
• kv must be torch.float8_e4m3fn, got {kv.dtype}
• indices must be contiguous
• indices must have shape ({s_q}, {h_kv}, topk), got {tuple(in
• attn_sink must be float32 with shape ({h_q},), got {tuple(at

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4e2439217c2a401b. Report an issue: GitHub.