sgl-project/sglang · error · ValueError

Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.

Error message

Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.

What it means

validate_packed_decode_inputs infers the query head count H = q_dim // K from mixed_qkv and requires H > 0 and HV % H == 0 (each query head maps to an integral group of value heads). This final check guarantees the head-layout is consistent before returning (B, H, HV, K, V) to the kernel launchers.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_decode.py:327

            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(
            f"Invalid packed Q size {q_dim}: must be divisible by K={K}. "
            "KDA packed decode requires num_q_heads == num_k_heads and "
            "head_q_dim == head_k_dim."
        )
    H = q_dim // K
    if H <= 0 or HV % H != 0:
        raise ValueError(
            f"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}."
        )
    return B, H, HV, K, V


def helion_fused_recurrent_kda_packed_decode(
    mixed_qkv: torch.Tensor,
    a: torch.Tensor,
    b: torch.Tensor,
    A_log: torch.Tensor,
    dt_bias: torch.Tensor,
    scale: float,
    initial_state: torch.Tensor,
    out: torch.Tensor,
    ssm_state_indices: torch.Tensor,
    use_qk_l2norm_in_kernel: bool = False,
    lower_bound: float | None = None,
) -> tuple[torch.Tensor, torch.Tensor]:

View on GitHub (pinned to 0132848349)

Solutions

  1. Make num_q_heads divide num_v_heads (or equal it) in the model config
  2. Recompute mixed_qkv from the actual projections instead of slicing/manually assembling it
  3. Unit-test the head inference: assert HV % (q_dim // K) == 0 in your model tests

Example fix

// before
cfg.num_q_heads = 3; cfg.num_v_heads = 4  # 4 % 3 != 0
// after
cfg.num_q_heads = 2; cfg.num_v_heads = 4  # 4 % 2 == 0
Defensive patterns

Strategy: validation

Validate before calling

HV, V, K = initial_state.shape[-3:]
q_dim = (mixed_qkv.shape[1] - HV * V) // 2
H = q_dim // K
assert H > 0 and HV % H == 0, (H, HV)

Type guard

def valid_head_layout(qkv: torch.Tensor, hv: int, v: int, k: int) -> bool:
    q_dim = (qkv.shape[1] - hv * v) // 2
    h = q_dim // k
    return q_dim % k == 0 and h > 0 and hv % h == 0

Prevention

When it happens

Trigger: A mixed_qkv whose Q width implies H that does not divide HV, e.g. H=3 with HV=4; degenerate q width yielding H=0 after the K-divisibility check passed via rounding.

Common situations: Hand-crafted test tensors with arbitrary widths; a model config where num_q_heads and num_v_heads are coprime; a corrupted projection weight producing an unexpected qkv width.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/015041fcc4da436a. Report an issue: GitHub.