sgl-project/sglang · error · ValueError

Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V

Error message

Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}.

What it means

mixed_qkv packs query, key and value projections along dim 1. After subtracting the value part (HV*V), the remaining qk width must be positive and even (Q and K equal width). validate_packed_decode_inputs raises when the remaining width is <= 0 or odd, meaning the packed layout is not the expected [q | k | v] split.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_decode.py:315

    if b.shape[1] != HV:
        raise ValueError(
            f"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)})."
        )
    if A_log.numel() != HV:
        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(
            f"Invalid packed Q size {q_dim}: must be divisible by K={K}. "
            "KDA packed decode requires num_q_heads == num_k_heads and "
            "head_q_dim == head_k_dim."
        )
    H = q_dim // K
    if H <= 0 or HV % H != 0:
        raise ValueError(
            f"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}."
        )
    return B, H, HV, K, V


def helion_fused_recurrent_kda_packed_decode(

View on GitHub (pinned to 0132848349)

Solutions

  1. Verify mixed_qkv.shape[1] == 2*num_q_heads*head_k_dim + num_v_heads*head_v_dim
  2. Ensure the qkv projection orders segments as [Q, K, V] with Q and K of equal total width
  3. If head_q_dim != head_k_dim, this packed kernel cannot be used — fall back to the unpacked path

Example fix

// before
qkv = self.qkv_proj(x)  # widths Q=2K... mismatched
// after
assert qkv.shape[1] == 2*H*K + HV*V
qkv = self.qkv_proj(x)
Defensive patterns

Strategy: validation

Validate before calling

HV, V = initial_state.shape[-3], initial_state.shape[-2]
qkv_dim = mixed_qkv.shape[1]
assert qkv_dim > HV * V and (qkv_dim - HV * V) % 2 == 0, qkv_dim

Type guard

def valid_packed_qkv(qkv: torch.Tensor, hv: int, v: int) -> bool:
    rest = qkv.shape[1] - hv * v
    return rest > 0 and rest % 2 == 0

Prevention

When it happens

Trigger: Passing mixed_qkv containing only q and v (no k), a qkv fused tensor with Q and K of different widths, or a tensor whose dim 1 includes an extra gate segment.

Common situations: Model projects q/k/v with head_q_dim != head_k_dim; a fused qkv linear whose output width was changed but the kernel expectation wasn't; feeding the wrong projection output into the fused decode.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/09a43ccdf753ab34. Report an issue: GitHub.