sgl-project/sglang · error · ValueError

Unsupported d_qk: {d_qk}. Expected {DSV4_D_QK} (DeepSeek V4)

Error message

Unsupported d_qk: {d_qk}. Expected {DSV4_D_QK} (DeepSeek V4)

What it means

triton_sparse_attn_decode is a DeepSeek V4-specialized kernel that hard-requires the DSV4 query/key head dimension (DSV4_D_QK = 512). It reads d_qk = q.shape[-1] and rejects anything else, forwarding only the DSV4 layout to _triton_sparse_attn_decode_dsv4.

Source

Thrown at python/sglang/kernels/ops/attention/nsa_triton_decode/triton_mla_kernels_decode_optimized.py:62

            return total_tokens <= 32
        else:
            return total_tokens <= 128
    return True


def triton_sparse_attn_decode(
    q: torch.Tensor,
    kv_scope,
    extra_kv_scope,
    sm_scale: float,
    d_v: int = 512,
    attn_sink: Optional[torch.Tensor] = None,
) -> Tuple[torch.Tensor, torch.Tensor]:
    """Optimized sparse attention decode for DeepSeek V4 (d_qk=512)."""
    d_qk = q.shape[-1]

    if d_qk != DSV4_D_QK:
        raise ValueError(
            f"Unsupported d_qk: {d_qk}. Expected {DSV4_D_QK} (DeepSeek V4)"
        )

    return _triton_sparse_attn_decode_dsv4(
        q, kv_scope, extra_kv_scope, sm_scale, d_v, attn_sink
    )


def _triton_sparse_attn_decode_dsv4(
    q: torch.Tensor,
    kv_scope,
    extra_kv_scope,
    sm_scale: float,
    d_v: int,
    attn_sink: Optional[torch.Tensor],
) -> Tuple[torch.Tensor, torch.Tensor]:
    """Sparse attention decode for DeepSeek V4 (d_qk=512).

View on GitHub (pinned to 0132848349)

Solutions

1. Route the model to its correct decode kernel (DSV3 or generic sparse attention path)
2. If you intend DSV4, check that q was projected to the 512-dim qk layout before this call

Example fix

# before
out = triton_sparse_attn_decode(q, kv_scope, ...)  # q.shape[-1] == 576
# after
if q.shape[-1] != DSV4_D_QK:
    out = dsv3_sparse_attn_decode(q, kv, ...)
else:
    out = triton_sparse_attn_decode(q, kv_scope, ...)

d_qk = q.shape[-1]
fn = triton_sparse_attn_decode if d_qk == DSV4_D_QK else generic_sparse_decode
out = fn(q, kv_scope, ...)

def is_dsv4_q(q: torch.Tensor) -> bool:
    return q.ndim == 3 and q.shape[-1] == 512  # DSV4_D_QK

When it happens

Trigger: Calling triton_sparse_attn_decode (directly or via triton_fp8_attention_fwd) with a q tensor whose last dimension is not 512 — e.g. DSV3's 576 or a standard 128-dim head.

Common situations: Running a non-DSV4 model (DeepSeek V3/R1, or a generic MLA model) while the attention backend selects the DSV4-optimized decode kernel; model-architecture dispatch falling through to the wrong kernel.

Related errors

• sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtyp
• Unsupported split_k: {split_k}
• {self._op_label()}: no triton backend
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• D={D_check} must be divisible by GROUP_SIZE={_FP8_GROUP_SIZE

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bcf9f96344626abb. Report an issue: GitHub.