sgl-project/sglang · error · ValueError

indices must be contiguous

Error message

indices must be contiguous

What it means

The sparse MLA q8kv8 prefill kernel requires contiguous indices. Call indices.contiguous() before invoking.

Solutions

1. Call indices = indices.contiguous() before the call
2. Avoid slicing/striding the indices buffer; build it at the exact (s_q, h_kv, topk) shape
3. Check intermediate ops (flip, narrow, expand) between topk and the kernel call

Example fix

// before
idx = torch.sort(indices, dim=-1).values  # may be non-contiguous
out = sparse_mla_q8kv8_prefill_fwd(q, kv, idx)
// after
idx = torch.sort(indices, dim=-1).values.contiguous()
out = sparse_mla_q8kv8_prefill_fwd(q, kv, idx)

if not indices.is_contiguous(): indices = indices.contiguous()

def indices_ready(indices: torch.Tensor) -> bool:
    return indices.is_contiguous() and indices.dtype == torch.int32

When it happens

Trigger: Passing indices produced by a top-k selection (torch.topk returns contiguous, but a subsequent slice/permute like indices[:, :, ::2] or indices.permute(...)) that is non-contiguous.

Common situations: Post-processing top-k output (sorting, deduplication, gather) with views; reusing a padded indices buffer sliced to actual topk; index tensors forwarded from a different attention path with different layout.

Related errors

• attn_sink must be contiguous
• indices must be int32, got
• indices must have shape
• q must be contiguous
• `a`/`b` must be contiguous in the last dim.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cadf788220b8c7ae. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:340

    if kv.device != device:
        raise ValueError(f"kv must be on q's device {device}, got {kv.device}")
    if indices.device != device:
        raise ValueError(
            f"indices must be on q's device {device}, got {indices.device}"
        )

    if q.dtype != torch.float8_e4m3fn:
        raise ValueError(f"q must be torch.float8_e4m3fn, got {q.dtype}")
    if kv.dtype != torch.float8_e4m3fn:
        raise ValueError(f"kv must be torch.float8_e4m3fn, got {kv.dtype}")

    if not q.is_contiguous():
        raise ValueError("q must be contiguous")
    if not kv.is_contiguous():
        raise ValueError("kv must be contiguous")
    if not indices.is_contiguous():
        raise ValueError("indices must be contiguous")

    if kv_d_qk != d_qk:
        raise ValueError(f"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}")

    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.
    # Reject unpadded TP-local head counts instead of launching zero CTAs and
    # returning uninitialized outputs, which can appear to callers as a hang or
    # a later collective failure.
    if h_q == 0 or h_q % 64 != 0:
        raise ValueError(
            "sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive "
            f"multiple of 64, got {h_q}"
        )

    if h_kv != 1:
        raise ValueError(f"sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}")

    if d_qk not in (512, 576):

View on GitHub (pinned to 0132848349)