{"record":{"id":"0b8dfbbe4e693f59","repo":"sgl-project/sglang","slug":"attn-sink-must-be-contiguous","errorCode":null,"errorMessage":"attn_sink must be contiguous","messagePattern":"attn_sink must be contiguous","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py","lineNumber":419,"sourceCode":"        )\n\n    if attn_sink is not None and topk_length is None:\n        raise ValueError(\"attn_sink requires topk_length to be provided as well\")\n\n    if attn_sink is not None:\n        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:\n            raise ValueError(\n                f\"attn_sink must be float32 with shape ({h_q},), got \"\n                f\"{tuple(attn_sink.shape)}/{attn_sink.dtype}\"\n            )\n        if not attn_sink.is_cuda:\n            raise ValueError(\"attn_sink must be a CUDA tensor\")\n        if attn_sink.device != device:\n            raise ValueError(\n                f\"attn_sink must be on q's device {device}, got {attn_sink.device}\"\n            )\n        if not attn_sink.is_contiguous():\n            raise ValueError(\"attn_sink must be contiguous\")\n\n    for name, scale in ((\"q_scale\", q_scale), (\"kv_scale\", kv_scale)):\n        if not isinstance(scale, torch.Tensor):\n            raise ValueError(f\"{name} must be a torch.Tensor\")\n        if not scale.is_cuda:\n            raise ValueError(f\"{name} must be a CUDA tensor\")\n        if scale.device != device:\n            raise ValueError(\n                f\"{name} must be on q's device {device}, got {scale.device}\"\n            )\n        if scale.dtype != torch.float32:\n            raise ValueError(f\"{name} must be float32, got {scale.dtype}\")\n        if scale.numel() != 1:\n            raise ValueError(\n                f\"{name} must be a scalar tensor, got shape {tuple(scale.shape)}\"\n            )\n        if not scale.is_contiguous():\n            raise ValueError(f\"{name} must be contiguous\")","sourceCodeStart":401,"sourceCodeEnd":437,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py#L401-L437","documentation":"The sparse MLA prefill kernel requires attn_sink to be contiguous so it can index it with a simple stride of 1 across heads. A non-contiguous view (e.g. a slice of a larger tensor with gaps) would make the raw pointer arithmetic wrong, so the wrapper rejects it before launch.","triggerScenarios":"Passing attn_sink = some_2d_tensor[i] where the row has stride > 1, or any tensor whose is_contiguous() is False due to slicing/transposition.","commonSituations":"Slicing the sink out of a fused parameter buffer (e.g. stacking sinks for several layers and indexing one row that is non-contiguous).","solutions":["Call attn_sink = attn_sink.contiguous() before passing it","Clone the slice when extracting it from a larger buffer"],"exampleFix":"// before\nattn_sink = stacked_sinks[:, layer_idx]  # may be non-contiguous\n// after\nattn_sink = stacked_sinks[:, layer_idx].contiguous()","handlingStrategy":"validation","validationCode":"attn_sink = attn_sink.contiguous() if not attn_sink.is_contiguous() else attn_sink","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Call .contiguous() when extracting per-layer values from stacked buffers","Prefer torch.empty + copy_ over fancy-indexed slices for kernel inputs"],"tags":["mla","sparse-attention","tensor-validation","contiguity"],"backgroundTag":"non-contiguous-tensor","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}