sgl-project/sglang · error · ValueError

Invalid packed Q size {q_dim}: must be divisible by K={K}. K

Error message

Invalid packed Q size {q_dim}: must be divisible by K={K}. KDA packed decode requires num_q_heads == num_k_heads and head_q_dim == head_k_dim.

What it means

After validating the even qk width, validate_packed_decode_inputs splits it in half to get the packed Q size and requires it to be a multiple of K (head_k_dim). This enforces the packed-decode constraint num_q_heads == num_k_heads and head_q_dim == head_k_dim, since the kernel indexes q as [H, K].

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_decode.py:320

        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(
            f"Invalid packed Q size {q_dim}: must be divisible by K={K}. "
            "KDA packed decode requires num_q_heads == num_k_heads and "
            "head_q_dim == head_k_dim."
        )
    H = q_dim // K
    if H <= 0 or HV % H != 0:
        raise ValueError(
            f"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}."
        )
    return B, H, HV, K, V


def helion_fused_recurrent_kda_packed_decode(
    mixed_qkv: torch.Tensor,
    a: torch.Tensor,
    b: torch.Tensor,
    A_log: torch.Tensor,
    dt_bias: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Fall back to a decode path that supports GQA, or repack q so num_q_heads == num_k_heads with head_q_dim == head_k_dim
2. Fix the model config: head_q_dim = head_k_dim and num_q_heads = num_k_heads
3. Assert (qkv.shape[1] - HV*V) % (2*K) == 0 before the call

Example fix

// before
# config: num_q_heads=32, num_k_heads=8 (GQA)
use_packed_helion = True
// after
use_packed_helion = (cfg.num_q_heads == cfg.num_k_heads and cfg.head_q_dim == cfg.head_k_dim)

HV, V, K = initial_state.shape[-3:]
q_dim = (mixed_qkv.shape[1] - HV * V) // 2
packed_ok = q_dim % K == 0 and HV % (q_dim // K) == 0
backend = "helion_packed" if packed_ok else "triton"

When it happens

Trigger: A GQA layout where num_q_heads > num_k_heads is packed into mixed_qkv and passed to helion_fused_recurrent_kda_packed_decode; head_q_dim set differently from head_k_dim in the model config.

Common situations: Reusing a GQA attention config with the KDA packed decode kernel; migrating a model whose q heads were duplicated (MQA-style) without repacking; test tensors built with q width not a multiple of K.

Related errors

• Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.
• `a` must have shape [B, HV*K] with HV={HV}, K={K} (got a.sha
• `b` must have shape [B, HV] with HV={HV} (got b.shape={tuple
• Invalid packed Q size {q_dim}: must be divisible by K={K}. K
• `mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9491e58ae1d432e3. Report an issue: GitHub.