sgl-project/sglang · error · ValueError

`a` must have shape [B, HV*K] with HV={HV}, K={K} (got a.sha

Error message

`a` must have shape [B, HV*K] with HV={HV}, K={K} (got a.shape={tuple(a.shape)}).

What it means

The gate tensor a must have width HV*K (per value-head, per-key-channel decay inputs). This error fires when a.shape[1] differs, meaning the gating projection width doesn't match the state's head/key config.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:588

    if a.shape[0] != B or b.shape[0] != B:
        raise ValueError(
            "Mismatched batch sizes: "
            f"mixed_qkv.shape[0]={B}, a.shape[0]={a.shape[0]}, b.shape[0]={b.shape[0]}."
        )
    if ssm_state_indices.shape[0] != B:
        raise ValueError(
            f"`ssm_state_indices` must have shape [B] (got {tuple(ssm_state_indices.shape)}; expected ({B},))."
        )

    if initial_state.ndim != 4:
        raise ValueError(
            f"`initial_state` must be a 4D tensor (got ndim={initial_state.ndim})."
        )
    if initial_state.stride(-1) != 1:
        raise ValueError("`initial_state` must be contiguous in the last dim.")
    HV, V, K = initial_state.shape[-3:]
    if a.shape[1] != HV * K:
        raise ValueError(
            f"`a` must have shape [B, HV*K] with HV={HV}, K={K} "
            f"(got a.shape={tuple(a.shape)})."
        )
    if b.shape[1] != HV:
        raise ValueError(
            f"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)})."
        )
    if A_log.numel() != HV:
        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Slice/build a with width HV*K from the gating projection
2. Confirm the split offsets of the fused projection match HV*K and HV
3. Assert a.shape == (B, HV*K) using dims from initial_state

Example fix

# before
a = proj[:, :H*K]  # used query heads
# after
a = proj[:, :HV*K]  # value heads times key dim

HV, V, K = initial_state.shape[-3:]
assert a.shape == (mixed_qkv.shape[0], HV * K), (a.shape, HV, K)

When it happens

Trigger: a has shape [B, H*K] with H != HV, or [B, HV] (missing the K factor), while initial_state implies HV*K columns.

Common situations: Using Q-head-count instead of V-head-count when slicing the a gate from a fused projection; model configs where num_heads != num_v_heads (GQA).

Related errors

• `b` must have shape [B, HV] with HV={HV} (got b.shape={tuple
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k
• k_pool has incompatible shape {k_pool.shape}
• Unexpected A_log shape: {A_log.shape}; expected numel={HV}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c5fa7fa0cc4d44ce. Report an issue: GitHub.