sgl-project/sglang · error · ValueError

attn_sink must be contiguous

Error message

attn_sink must be contiguous

What it means

The sparse MLA prefill kernel requires attn_sink to be contiguous so it can index it with a simple stride of 1 across heads. A non-contiguous view (e.g. a slice of a larger tensor with gaps) would make the raw pointer arithmetic wrong, so the wrapper rejects it before launch.

Source

Thrown at python/sglang/kernels/ops/attention/sparse_mla_q8kv8_prefill_sm90.py:419

        )

    if attn_sink is not None and topk_length is None:
        raise ValueError("attn_sink requires topk_length to be provided as well")

    if attn_sink is not None:
        if attn_sink.shape != (h_q,) or attn_sink.dtype != torch.float32:
            raise ValueError(
                f"attn_sink must be float32 with shape ({h_q},), got "
                f"{tuple(attn_sink.shape)}/{attn_sink.dtype}"
            )
        if not attn_sink.is_cuda:
            raise ValueError("attn_sink must be a CUDA tensor")
        if attn_sink.device != device:
            raise ValueError(
                f"attn_sink must be on q's device {device}, got {attn_sink.device}"
            )
        if not attn_sink.is_contiguous():
            raise ValueError("attn_sink must be contiguous")

    for name, scale in (("q_scale", q_scale), ("kv_scale", kv_scale)):
        if not isinstance(scale, torch.Tensor):
            raise ValueError(f"{name} must be a torch.Tensor")
        if not scale.is_cuda:
            raise ValueError(f"{name} must be a CUDA tensor")
        if scale.device != device:
            raise ValueError(
                f"{name} must be on q's device {device}, got {scale.device}"
            )
        if scale.dtype != torch.float32:
            raise ValueError(f"{name} must be float32, got {scale.dtype}")
        if scale.numel() != 1:
            raise ValueError(
                f"{name} must be a scalar tensor, got shape {tuple(scale.shape)}"
            )
        if not scale.is_contiguous():
            raise ValueError(f"{name} must be contiguous")

View on GitHub (pinned to 0132848349)

Solutions

1. Call attn_sink = attn_sink.contiguous() before passing it
2. Clone the slice when extracting it from a larger buffer

Example fix

// before
attn_sink = stacked_sinks[:, layer_idx]  # may be non-contiguous
// after
attn_sink = stacked_sinks[:, layer_idx].contiguous()

attn_sink = attn_sink.contiguous() if not attn_sink.is_contiguous() else attn_sink

When it happens

Trigger: Passing attn_sink = some_2d_tensor[i] where the row has stride > 1, or any tensor whose is_contiguous() is False due to slicing/transposition.

Common situations: Slicing the sink out of a fused parameter buffer (e.g. stacking sinks for several layers and indexing one row that is non-contiguous).

Related errors

• attn_sink must be float32 with shape ({h_q},), got {tuple(at
• attn_sink must be a CUDA tensor
• q must be contiguous
• indices must be contiguous
• sparse_mla_q8kv8_prefill_fwd requires h_kv=1, got {h_kv}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0b8dfbbe4e693f59. Report an issue: GitHub.