sgl-project/sglang · error · ValueError

`A_log` must have {HV} elements (got {A_log.numel()}).

Error message

`A_log` must have {HV} elements (got {A_log.numel()}).

What it means

A_log holds the log-decay coefficients, one per value head, so validate_packed_decode_inputs requires exactly HV elements (any shape). The numel check catches A_log tensors sized for a different head count, e.g. per-key-dim or query-head-sized decay parameters.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_decode.py:302

    if initial_state.stride(-1) != 1:
        raise ValueError("`initial_state` must be contiguous in the last dim.")
    HV, V, K = initial_state.shape[-3:]
    if not _is_power_of_two(K) or not _is_power_of_two(V):
        raise ValueError(
            "Helion KDA decode requires power-of-two key and value head "
            f"dimensions (got K={K}, V={V})."
        )
    if a.shape[1] != HV * K:
        raise ValueError(
            f"`a` must have shape [B, HV*K] with HV={HV}, K={K} "
            f"(got a.shape={tuple(a.shape)})."
        )
    if b.shape[1] != HV:
        raise ValueError(
            f"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)})."
        )
    if A_log.numel() != HV:
        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(

View on GitHub (pinned to 0132848349)

Solutions

1. Verify A_log comes from the model parameter of shape [num_v_heads] and pass it unmodified
2. If the checkpoint has a different size, fix the model config / re-load the correct weights rather than padding
3. In tests, size A_log as torch.empty(HV) not (HV, K)

Example fix

// before
A_log = layer.A_log.expand(HV, K).contiguous()  # numel = HV*K
// after
A_log = layer.A_log  # shape [HV], numel == HV

HV = initial_state.shape[-3]
assert A_log.numel() == HV, (A_log.shape, HV)

def valid_a_log(t: torch.Tensor, hv: int) -> bool:
    return t.numel() == hv

When it happens

Trigger: Passing A_log with numel == HV*K (duplicated per key dim), numel == H (query heads, GQA mismatch), or a stale checkpoint parameter sized for another layer config.

Common situations: Loading a KDA checkpoint whose num_v_heads changed between model revisions; broadcasting A_log in tests with torch.full((HV, K), ...); mixing up A_log with dt_bias sizing.

Related errors

• `dt_bias` must have {HV * K} elements (got {dt_bias.numel()}
• Helion KDA decode requires power-of-two key and value head d
• `mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).
• `mixed_qkv` must be contiguous in the last dim.
• `a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim=

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1b202cdd305a7080. Report an issue: GitHub.