sgl-project/sglang · error · ValueError

`A_log` must have {HV} elements (got {A_log.numel()}).

Error message

`A_log` must have {HV} elements (got {A_log.numel()}).

What it means

A_log must contain exactly HV elements (one log-decay per value head) to match the state's head configuration. This fires when A_log has the query head count, or was loaded/reshaped incorrectly.

Source

Thrown at python/sglang/kernels/ops/attention/fla/fused_recurrent.py:597

    if initial_state.ndim != 4:
        raise ValueError(
            f"`initial_state` must be a 4D tensor (got ndim={initial_state.ndim})."
        )
    if initial_state.stride(-1) != 1:
        raise ValueError("`initial_state` must be contiguous in the last dim.")
    HV, V, K = initial_state.shape[-3:]
    if a.shape[1] != HV * K:
        raise ValueError(
            f"`a` must have shape [B, HV*K] with HV={HV}, K={K} "
            f"(got a.shape={tuple(a.shape)})."
        )
    if b.shape[1] != HV:
        raise ValueError(
            f"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)})."
        )
    if A_log.numel() != HV:
        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Use the per-layer, per-value-head parameter: A_log = model.A_log[layer_idx] with numel == HV
2. Verify the checkpoint's A_log shape matches model config num_v_heads
3. Assert A_log.numel() == initial_state.shape[-3] before decode

Example fix

# before
A_log = all_layers_A_log  # [L, HV]
# after
A_log = all_layers_A_log[layer_idx]  # [HV]

HV = initial_state.shape[-3]
assert A_log.numel() == HV, (A_log.shape, HV)

When it happens

Trigger: A_log.numel() != HV, e.g. a [H]-sized A_log in a GQA model (H != HV), or a flattened multi-layer A_log of size L*HV.

Common situations: Checkpoint with per-query-head A_log; passing a stacked all-layer parameter tensor instead of the current layer's slice.

Related errors

• Unexpected A_log shape: {A_log.shape}; expected numel={HV}
• Unexpected dt_bias shape: {dt_bias.shape}; expected numel={H
• Unexpected a shape for varlen: {a.shape}
• Unexpected a shape for dense: {a.shape}
• `a` must have shape [B, HV*K] with HV={HV}, K={K} (got a.sha

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/18e7c59e3cf0d8a8. Report an issue: GitHub.