sgl-project/sglang · error · ValueError

q must be contiguous

Error message

q must be contiguous

What it means

sparse_mla_q8kv8_prefill_fwd requires q to be a contiguous (dense, row-major) tensor because the CUDA kernel indexes q with raw pointer arithmetic assuming a contiguous layout. Non-contiguous q (e.g. a sliced or transposed view) is rejected before launch.

Source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:336

    if not kv.is_cuda:
        raise ValueError("kv must be a CUDA tensor")
    if not indices.is_cuda:
        raise ValueError("indices must be a CUDA tensor")

    if kv.device != device:
        raise ValueError(f"kv must be on q's device {device}, got {kv.device}")
    if indices.device != device:
        raise ValueError(
            f"indices must be on q's device {device}, got {indices.device}"
        )

    if q.dtype != torch.float8_e4m3fn:
        raise ValueError(f"q must be torch.float8_e4m3fn, got {q.dtype}")
    if kv.dtype != torch.float8_e4m3fn:
        raise ValueError(f"kv must be torch.float8_e4m3fn, got {kv.dtype}")

    if not q.is_contiguous():
        raise ValueError("q must be contiguous")
    if not kv.is_contiguous():
        raise ValueError("kv must be contiguous")
    if not indices.is_contiguous():
        raise ValueError("indices must be contiguous")

    if kv_d_qk != d_qk:
        raise ValueError(f"kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}")

    # The CUDA implementation uses B_H=64 and launches h_q / B_H CTAs.
    # Reject unpadded TP-local head counts instead of launching zero CTAs and
    # returning uninitialized outputs, which can appear to callers as a hang or
    # a later collective failure.
    if h_q == 0 or h_q % 64 != 0:
        raise ValueError(
            "sparse_mla_q8kv8_prefill_fwd requires h_q padded to a positive "
            f"multiple of 64, got {h_q}"
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Call q = q.contiguous() before invoking the kernel
2. If slicing caused it, materialize the slice with .clone() or rework the projection to output contiguous tensors
3. Audit any view/permute applied to q between the linear layer and the attention call

Example fix

// before
out = sparse_mla_q8kv8_prefill_fwd(q[:, :, :576].contiguous-like_view, ...)
// after
q = q[:, :, :576].contiguous()
out = sparse_mla_q8kv8_prefill_fwd(q, ...)

if not q.is_contiguous(): q = q.contiguous()

def contiguous_or_fix(t: torch.Tensor) -> torch.Tensor:
    return t if t.is_contiguous() else t.contiguous()

When it happens

Trigger: Passing q as a slice like q[:, :, :512] or a permuted/transposed view that yields is_contiguous()==False.

Common situations: Splitting head dims (d_qk 512 vs 576 + d_v 512) by slicing; using q.transpose(...) or narrow views from a fused projection output; per-token FP8 quantization helpers that return strided views.

Related errors

• indices must be on q's device {device}, got {indices.device}
• kv must be contiguous
• indices must be contiguous
• attn_sink must be contiguous
• unsupported input for causal Conv3D cat/pad CUDA

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/6639f6aa31b4dd8d. Report an issue: GitHub.