sgl-project/sglang · error · ValueError

topk_length values must satisfy 0 <= topk_length <= topk ({t

Error message

topk_length values must satisfy 0 <= topk_length <= topk ({topk})

What it means

Each entry of topk_length (the effective per-token sparse length) must be within [0, topk], where topk is the padded width of indices' last dim. Negative values or values exceeding the allocated index width would make the kernel read out-of-bounds KV entries.

Source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:394

        )

    if topk_length is not None:
        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:
            raise ValueError(
                f"topk_length must be int32 with shape ({s_q},), got "
                f"{tuple(topk_length.shape)}/{topk_length.dtype}"
            )
        if not topk_length.is_cuda:
            raise ValueError("topk_length must be a CUDA tensor")
        if topk_length.device != device:
            raise ValueError(
                "topk_length must be on q's device "
                f"{device}, got {topk_length.device}"
            )
        if not topk_length.is_contiguous():
            raise ValueError("topk_length must be contiguous")
        if torch.any(topk_length < 0).item() or torch.any(topk_length > topk).item():
            raise ValueError(
                "topk_length values must satisfy " f"0 <= topk_length <= topk ({topk})"
            )

    if d_v != 512:
        raise ValueError(
            f"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v}"
        )

    if attn_sink is not None and topk_length is None:
        raise ValueError("attn_sink requires topk_length to be provided as well")

    if attn_sink is not None:
        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:
            raise ValueError(
                f"attn_sink must be float32 with shape ({h_q},), got "
                f"{tuple(attn_sink.shape)}/{attn_sink.dtype}"
            )
        if not attn_sink.is_cuda:

View on GitHub (pinned to 0132848349)

Solutions

1. Clamp before the call: topk_length = topk_length.clamp_(0, indices.shape[-1])
2. Recompute lengths against the padded topk width, not the raw selection count
3. Validate the producer of topk_length writes initialized values in [0, topk]

Example fix

// before
out = fwd(q, kv, indices, topk_length=lengths)  # some lengths == 200, topk=128
// after
lengths = lengths.clamp_(0, indices.shape[-1])
out = fwd(q, kv, indices, topk_length=lengths)

topk = indices.shape[-1]
assert torch.all((topk_length >= 0) & (topk_length <= topk)).item()

def lengths_in_range(indices: torch.Tensor, tl: torch.Tensor) -> bool:
    return bool(((tl >= 0) & (tl <= indices.shape[-1])).all().item())

When it happens

Trigger: topk_length containing topk+1 or a negative count; using the padded width (e.g. 128) in indices but lengths computed against a larger true selection size (e.g. 204).

Common situations: Padding indices to a multiple of 128 while forgetting to clamp effective lengths to that padded width; off-by-one when lengths represent 'last valid index + 1'; corrupted/uninitialized length buffers from a producer kernel.

Related errors

• Q8KV8 sparse-prefill topk width must be a positive multiple
• bad compress_ratio {compress_ratio}
• indices must have shape (s_q, h_kv, topk), got {tuple(indice
• indices must be on q's device {device}, got {indices.device}
• q must be torch.float8_e4m3fn, got {q.dtype}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/96c7a8aeefa9cb3c. Report an issue: GitHub.