sgl-project/sglang · error · ValueError

Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.

Error message

Invalid head config inferred from mixed_qkv: H={H}, HV={HV}.

What it means

The head count H inferred as q_dim//K must be positive and must evenly divide HV (the number of value heads), since the kernel maps each value head to a group of query heads (GQA-style). If HV % H != 0, the query-to-value head grouping is invalid.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:352

            f"`A_log` and `dt_bias` must have {HV} elements (got A_log.numel()={A_log.numel()}, dt_bias.numel()={dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(f"Invalid packed Q size {q_dim}: must be divisible by K={K}.")
    H = q_dim // K
    if H <= 0 or HV % H != 0:
        raise ValueError(
            f"Invalid head config inferred from mixed_qkv: H={H}, HV={HV}."
        )

    BK = triton.next_power_of_2(K)
    if triton.cdiv(K, BK) != 1:
        raise ValueError(
            f"Packed decode kernel only supports NK=1 (got K={K}, BK={BK})."
        )
    BV = min(triton.next_power_of_2(V), 32)
    num_stages = 3
    num_warps = 1

    stride_mixed_qkv_tok = mixed_qkv.stride(0)
    stride_a_tok = a.stride(0)
    stride_b_tok = b.stride(0)
    stride_init_state_token = initial_state.stride(0)
    stride_final_state_token = initial_state.stride(0)
    stride_indices_seq = ssm_state_indices.stride(0)

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure num_query_heads divides num_v_heads (standard GQA ratio)
2. Check mixed_qkv.shape[1] is large enough that q_dim//K >= 1
3. Align the model's num_attention_heads / num_key_value_heads with the packed layout

H = ((mixed_qkv.shape[1] - HV * V) // 2) // K
assert H > 0 and HV % H == 0, (H, HV)

When it happens

Trigger: Calling packed decode with a config where num_q_heads does not divide num_v_heads (e.g. H=7, HV=4), or where inferred H<=0 due to a tiny mixed_qkv.

Common situations: Custom model configs with non-GQA head ratios, or a truncated mixed_qkv tensor making q_dim smaller than K.

Related errors

• Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V
• Invalid packed Q size {q_dim}: must be divisible by K={K}.
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k
• k_pool has incompatible shape {k_pool.shape}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d40ac57efaaab279. Report an issue: GitHub.