{"record":{"id":"96c7a8aeefa9cb3c","repo":"sgl-project/sglang","slug":"topk-length-values-must-satisfy-0-topk-length","errorCode":null,"errorMessage":"topk_length values must satisfy 0 <= topk_length <= topk ({topk})","messagePattern":"topk_length values must satisfy 0 <= topk_length <= topk \\((.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":394,"sourceCode":"        )\n\n    if topk_length is not None:\n        if topk_length.shape != (s_q,) or topk_length.dtype != torch.int32:\n            raise ValueError(\n                f\"topk_length must be int32 with shape ({s_q},), got \"\n                f\"{tuple(topk_length.shape)}/{topk_length.dtype}\"\n            )\n        if not topk_length.is_cuda:\n            raise ValueError(\"topk_length must be a CUDA tensor\")\n        if topk_length.device != device:\n            raise ValueError(\n                \"topk_length must be on q's device \"\n                f\"{device}, got {topk_length.device}\"\n            )\n        if not topk_length.is_contiguous():\n            raise ValueError(\"topk_length must be contiguous\")\n        if torch.any(topk_length < 0).item() or torch.any(topk_length > topk).item():\n            raise ValueError(\n                \"topk_length values must satisfy \" f\"0 <= topk_length <= topk ({topk})\"\n            )\n\n    if d_v != 512:\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v}\"\n        )\n\n    if attn_sink is not None and topk_length is None:\n        raise ValueError(\"attn_sink requires topk_length to be provided as well\")\n\n    if attn_sink is not None:\n        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:\n            raise ValueError(\n                f\"attn_sink must be float32 with shape ({h_q},), got \"\n                f\"{tuple(attn_sink.shape)}/{attn_sink.dtype}\"\n            )\n        if not attn_sink.is_cuda:","sourceCodeStart":376,"sourceCodeEnd":412,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L376-L412","documentation":"Each entry of topk_length (the effective per-token sparse length) must be within [0, topk], where topk is the padded width of indices' last dim. Negative values or values exceeding the allocated index width would make the kernel read out-of-bounds KV entries.","triggerScenarios":"topk_length containing topk+1 or a negative count; using the padded width (e.g. 128) in indices but lengths computed against a larger true selection size (e.g. 204).","commonSituations":"Padding indices to a multiple of 128 while forgetting to clamp effective lengths to that padded width; off-by-one when lengths represent 'last valid index + 1'; corrupted/uninitialized length buffers from a producer kernel.","solutions":["Clamp before the call: topk_length = topk_length.clamp_(0, indices.shape[-1])","Recompute lengths against the padded topk width, not the raw selection count","Validate the producer of topk_length writes initialized values in [0, topk]"],"exampleFix":"// before\nout = fwd(q, kv, indices, topk_length=lengths)  # some lengths == 200, topk=128\n// after\nlengths = lengths.clamp_(0, indices.shape[-1])\nout = fwd(q, kv, indices, topk_length=lengths)","handlingStrategy":"validation","validationCode":"topk = indices.shape[-1]\nassert torch.all((topk_length >= 0) & (topk_length <= topk)).item()","typeGuard":"def lengths_in_range(indices: torch.Tensor, tl: torch.Tensor) -> bool:\n    return bool(((tl >= 0) & (tl <= indices.shape[-1])).all().item())","tryCatchPattern":null,"preventionTips":["Clamp lengths to the padded topk width whenever you pad indices","Treat lengths as half-open counts [0, topk] and test boundaries"],"tags":["value-validation","topk","sparse-attention"],"backgroundTag":"value-out-of-range","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}