{"record":{"id":"fd02b156da0a1ab1","repo":"sgl-project/sglang","slug":"attn-sink-requires-topk-length-to-be-provided-as-w","errorCode":null,"errorMessage":"attn_sink requires topk_length to be provided as well","messagePattern":"attn_sink requires topk_length to be provided as well","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":404,"sourceCode":"        if topk_length.device != device:\n            raise ValueError(\n                \"topk_length must be on q's device \"\n                f\"{device}, got {topk_length.device}\"\n            )\n        if not topk_length.is_contiguous():\n            raise ValueError(\"topk_length must be contiguous\")\n        if torch.any(topk_length < 0).item() or torch.any(topk_length > topk).item():\n            raise ValueError(\n                \"topk_length values must satisfy \" f\"0 <= topk_length <= topk ({topk})\"\n            )\n\n    if d_v != 512:\n        raise ValueError(\n            f\"sparse_mla_q8kv8_prefill_fwd only supports d_v=512, got {d_v}\"\n        )\n\n    if attn_sink is not None and topk_length is None:\n        raise ValueError(\"attn_sink requires topk_length to be provided as well\")\n\n    if attn_sink is not None:\n        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:\n            raise ValueError(\n                f\"attn_sink must be float32 with shape ({h_q},), got \"\n                f\"{tuple(attn_sink.shape)}/{attn_sink.dtype}\"\n            )\n        if not attn_sink.is_cuda:\n            raise ValueError(\"attn_sink must be a CUDA tensor\")\n        if attn_sink.device != device:\n            raise ValueError(\n                f\"attn_sink must be on q's device {device}, got {attn_sink.device}\"\n            )\n        if not attn_sink.is_contiguous():\n            raise ValueError(\"attn_sink must be contiguous\")\n\n    for name, scale in ((\"q_scale\", q_scale), (\"kv_scale\", kv_scale)):\n        if not isinstance(scale, torch.Tensor):","sourceCodeStart":386,"sourceCodeEnd":422,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L386-L422","documentation":"The optional attention-sink parameter only makes sense with variable-length sparse masks: attn_sink requires topk_length to also be passed. Passing attn_sink without topk_length is rejected because the sink correction is defined relative to per-token masked lengths.","triggerScenarios":"Calling sparse_mla_q8kv8_prefill_fwd(..., attn_sink=sink) without a topk_length argument (e.g. mirroring a decode-path signature that took attn_sink alone).","commonSituations":"Adding attention-sink support to a new model integration but reusing the fixed-topk call site; API drift where an older/newer signature allowed attn_sink standalone.","solutions":["Also pass topk_length (int32, shape (s_q,), values in [0, topk]) when enabling attn_sink","If you have no variable lengths, pass topk_length full of the padded topk value to emulate fixed-length behavior","Or drop attn_sink if sinks are not needed for this call"],"exampleFix":"# before\nout = fwd(q, kv, indices, attn_sink=sink)\n# after\nlengths = torch.full((s_q,), indices.shape[-1], dtype=torch.int32, device=q.device)\nout = fwd(q, kv, indices, topk_length=lengths, attn_sink=sink)","handlingStrategy":"validation","validationCode":"if attn_sink is not None:\n    assert topk_length is not None, \"attn_sink requires topk_length\"","typeGuard":"def sink_args_valid(attn_sink, topk_length) -> bool:\n    return attn_sink is None or topk_length is not None","tryCatchPattern":null,"preventionTips":["Bind attn_sink and topk_length together in your call wrapper","Provide a fixed-length topk_length when enabling sinks without variable lengths"],"tags":["argument-validation","attention-sink","sparse-attention"],"backgroundTag":"missing-required-argument","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}