sgl-project/sglang · error · ValueError

`dt_bias` must have {HV * K} elements (got {dt_bias.numel()}

Error message

`dt_bias` must have {HV * K} elements (got {dt_bias.numel()}).

What it means

dt_bias supplies the softplus bias for the discretization, one value per (value-head, key-dim) pair, so validate_packed_decode_inputs requires exactly HV*K elements. The check mirrors the packing of `a` and catches dt_bias tensors sized per-head only or for the wrong head count.

Source

Thrown at python/sglang/kernels/ops/attention/helion/kda_decode.py:304

    HV, V, K = initial_state.shape[-3:]
    if not _is_power_of_two(K) or not _is_power_of_two(V):
        raise ValueError(
            "Helion KDA decode requires power-of-two key and value head "
            f"dimensions (got K={K}, V={V})."
        )
    if a.shape[1] != HV * K:
        raise ValueError(
            f"`a` must have shape [B, HV*K] with HV={HV}, K={K} "
            f"(got a.shape={tuple(a.shape)})."
        )
    if b.shape[1] != HV:
        raise ValueError(
            f"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)})."
        )
    if A_log.numel() != HV:
        raise ValueError(f"`A_log` must have {HV} elements (got {A_log.numel()}).")
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()})."
        )
    if out.shape != (B, 1, HV, V):
        raise ValueError(
            f"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)})."
        )

    qkv_dim = mixed_qkv.shape[1]
    qk_dim = qkv_dim - HV * V
    if qk_dim <= 0 or qk_dim % 2 != 0:
        raise ValueError(
            f"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}."
        )
    q_dim = qk_dim // 2
    if q_dim % K != 0:
        raise ValueError(
            f"Invalid packed Q size {q_dim}: must be divisible by K={K}. "
            "KDA packed decode requires num_q_heads == num_k_heads and "

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure dt_bias is flattened: dt_bias = dt_bias.reshape(-1) with numel == HV*K
2. Confirm the loaded checkpoint's dt_bias shape matches [num_v_heads, head_k_dim]
3. Match the packing of `a` — both must use the same HV and K inferred from initial_state

Example fix

// before
dt_bias = layer.dt_bias  # [HV] by mistake
// after
dt_bias = layer.dt_bias.reshape(-1)  # [HV*K] from [HV, K] storage

HV, V, K = initial_state.shape[-3:]
dt_bias = dt_bias.reshape(-1)
assert dt_bias.numel() == HV * K

def valid_dt_bias(t: torch.Tensor, hv: int, k: int) -> bool:
    return t.numel() == hv * k

When it happens

Trigger: Passing dt_bias with numel == HV (per-head bias), numel == H*K (query heads in a GQA layout), or flattened with an extra time dimension of size > 1.

Common situations: Checkpoint parameter renamed/reshaped between versions; test fixtures reusing A_log's shape for dt_bias; models where dt_bias is stored [HV, K] but sliced incorrectly.

Related errors

• `A_log` must have {HV} elements (got {A_log.numel()}).
• Helion KDA decode requires power-of-two key and value head d
• `mixed_qkv` must be a 2D tensor (got ndim={mixed_qkv.ndim}).
• `mixed_qkv` must be contiguous in the last dim.
• `a` and `b` must be 2D tensors (got a.ndim={a.ndim}, b.ndim=

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b697445ef9ffade0. Report an issue: GitHub.