sgl-project/sglang · error · ValueError

topk_length must be contiguous

Error message

topk_length must be contiguous

What it means

topk_length must be contiguous, like the other device tensors: the kernel reads it with raw pointer arithmetic over the (s_q,) int32 vector, so strided views are rejected.

Source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:392

            "Q8KV8 sparse-prefill topk width must be a positive multiple of 128, "
            f"got {topk}"
        )

    if topk_length is not None:
        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:
            raise ValueError(
                f"topk_length must be int32 with shape ({s_q},), got "
                f"{tuple(topk_length.shape)}/{topk_length.dtype}"
            )
        if not topk_length.is_cuda:
            raise ValueError("topk_length must be a CUDA tensor")
        if topk_length.device != device:
            raise ValueError(
                "topk_length must be on q's device "
                f"{device}, got {topk_length.device}"
            )
        if not topk_length.is_contiguous():
            raise ValueError("topk_length must be contiguous")
        if torch.any(topk_length < 0).item() or torch.any(topk_length > topk).item():
            raise ValueError(
                "topk_length values must satisfy " f"0 <= topk_length <= topk ({topk})"
            )

    if d_v != 512:
        raise ValueError(
            f"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v}"
        )

    if attn_sink is not None and topk_length is None:
        raise ValueError("attn_sink requires topk_length to be provided as well")

    if attn_sink is not None:
        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:
            raise ValueError(
                f"attn_sink must be float32 with shape ({h_q},), got "
                f"{tuple(attn_sink.shape)}/{attn_sink.dtype}"

View on GitHub (pinned to 0132848349)

Solutions

1. topk_length = topk_length.contiguous() before the call
2. Store lengths in a dedicated dense (s_q,) buffer
3. Avoid views/strides when assembling the lengths tensor

Example fix

// before
out = fwd(q, kv, indices, topk_length=lengths[start:start+s_q])
// after
topk_length = lengths[start:start+s_q].contiguous()
out = fwd(q, kv, indices, topk_length=topk_length)

if not topk_length.is_contiguous(): topk_length = topk_length.contiguous()

def dense_lengths(tl: torch.Tensor) -> bool:
    return tl.is_contiguous()

When it happens

Trigger: Passing a sliced topk_length (e.g. lengths[keep_mask] or a strided view of a larger buffer) that is non-contiguous.

Common situations: Slicing a pooled per-batch lengths buffer; boolean-mask indexing that happened to produce a copy vs. a view inconsistency; interleaving lengths with other metadata in one tensor.

Related errors

• `mixed_qkv` must be contiguous in the last dim.
• `a`/`b` must be contiguous in the last dim.
• `A_log`/`dt_bias` must be contiguous.
• `out` must be contiguous.
• `initial_state` must be contiguous in the last dim.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a7219ab00b9f0fe2. Report an issue: GitHub.