sgl-project/sglang · error · ValueError

indices must be contiguous

Error message

indices must be contiguous

What it means

The indices tensor (per-query top-k KV indices) must be contiguous int32 on the right device; this check rejects strided/sliced indices tensors because the kernel reads them with raw pointers.

Source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:340

    if kv.device != device:
        raise ValueError(f"kv must be on q's device {device}, got {kv.device}")
    if indices.device != device:
        raise ValueError(
            f"indices must be on q's device {device}, got {indices.device}"
        )

    if q.dtype != torch.float8_e4m3fn:
        raise ValueError(f"q must be torch.float8_e4m3fn, got {q.dtype}")
    if kv.dtype != torch.float8_e4m3fn:
        raise ValueError(f"kv must be torch.float8_e4m3fn, got {kv.dtype}")

    if not q.is_contiguous():
        raise ValueError("q must be contiguous")
    if not kv.is_contiguous():
        raise ValueError("kv must be contiguous")
    if not indices.is_contiguous():
        raise ValueError("indices must be contiguous")

    if kv_d_qk != d_qk:
        raise ValueError(f"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}")

    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.
    # Reject unpadded TP-local head counts instead of launching zero CTAs and
    # returning uninitialized outputs, which can appear to callers as a hang or
    # a later collective failure.
    if h_q == 0 or h_q % 64 != 0:
        raise ValueError(
            "sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive "
            f"multiple of 64, got {h_q}"
        )

    if h_kv != 1:
        raise ValueError(f"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}")

    if d_qk not in (512, 576):

View on GitHub (pinned to 0132848349)

Solutions

1. Call indices = indices.contiguous() before the call
2. Avoid slicing/striding the indices buffer; build it at the exact (s_q, h_kv, topk) shape
3. Check intermediate ops (flip, narrow, expand) between topk and the kernel call

Example fix

// before
idx = torch.sort(indices, dim=-1).values  # may be non-contiguous
out = sparse_mla_q8kv8_prefill_fwd(q, kv, idx)
// after
idx = torch.sort(indices, dim=-1).values.contiguous()
out = sparse_mla_q8kv8_prefill_fwd(q, kv, idx)

if not indices.is_contiguous(): indices = indices.contiguous()

def indices_ready(indices: torch.Tensor) -> bool:
    return indices.is_contiguous() and indices.dtype == torch.int32

When it happens

Trigger: Passing indices produced by a top-k selection (torch.topk returns contiguous, but a subsequent slice/permute like indices[:, :, ::2] or indices.permute(...)) that is non-contiguous.

Common situations: Post-processing top-k output (sorting, deduplication, gather) with views; reusing a padded indices buffer sliced to actual topk; index tensors forwarded from a different attention path with different layout.

Related errors

• q must be contiguous
• indices must have shape ({s_q}, {h_kv}, topk), got {tuple(in
• indices must be int32, got {indices.dtype}
• attn_sink must be contiguous
• bad compress_ratio {compress_ratio}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cadf788220b8c7ae. Report an issue: GitHub.