sgl-project/sglang · error · ValueError

Unexpected dt_bias shape: {dt_bias.shape}; expected numel={H

Error message

Unexpected dt_bias shape: {dt_bias.shape}; expected numel={HV * K}

What it means

_normalize_dt_bias requires the dt (delta-time) bias to have exactly HV*K elements so it can be reshaped to (HV, K). A different element count raises ValueError listing the expected numel.

Source

Thrown at python/sglang/kernels/ops/attention/cutedsl_kda.py:1353

    _compiled_kernels[key] = compiled_kernel
    logger.info(
        "CuTe DSL KDA kernel compiled: "
        f"N={N}, H={H}, HV={HV}, K={K}, V={V}, pool_size={pool_size}, "
        f"pool_strides={tuple(h0_source.stride())}, "
        f"small_batch={use_small_batch}, varlen={is_varlen_decode}"
    )
    return compiled_kernel


def _normalize_A_log(A_log: torch.Tensor, HV: int) -> torch.Tensor:
    if A_log.numel() != HV:
        raise ValueError(f"Unexpected A_log shape: {A_log.shape}; expected numel={HV}")
    return A_log.reshape(HV).contiguous()


def _normalize_dt_bias(dt_bias: torch.Tensor, HV: int, K: int) -> torch.Tensor:
    if dt_bias.numel() != HV * K:
        raise ValueError(
            f"Unexpected dt_bias shape: {dt_bias.shape}; expected numel={HV * K}"
        )
    return dt_bias.reshape(HV, K).contiguous()


def _normalize_kda_a(a, *, is_varlen_decode, N, HV, K):
    """Normalize `a` to match the compile-time shape expected by the kernel.

    varlen kernel compiled shape: (N, HV, K)  -- 3D
    dense kernel compiled shape:  (N, 1, HV, K) -- 4D
    """
    if is_varlen_decode:
        # Target: (N, HV, K) -- 3D
        if a.dim() == 2 and a.shape == (N, HV * K):
            return a.view(N, HV, K)
        if a.dim() == 3 and a.shape == (N, HV, K):
            return a  # already correct
        if a.dim() == 4 and a.shape == (1, N, HV, K):

View on GitHub (pinned to 0132848349)

Solutions

1. Align HV/K arguments with the checkpoint: HV*K must equal dt_bias.numel()
2. Reshape dt_bias to (HV, K) once at load time so the guard passes trivially
3. Add a load-time assertion: assert dt_bias.numel() == HV * K

Example fix

# before: dt_bias shape (num_heads,) but HV*K expected -> ValueError
# after
dt_bias = checkpoint_dt_bias.reshape(HV, K).contiguous()
update = cutedsl_fused_sigmoid_gating_kda_update(A_log, dt_bias, q, k, v, ...)

assert dt_bias.numel() == HV * K, f'dt_bias numel {dt_bias.numel()} != HV*K {HV*K}'

def is_valid_dt_bias(dt_bias, HV: int, K: int) -> bool:
    return dt_bias.numel() == HV * K

When it happens

Trigger: Calling cutedsl_fused_sigmoid_gating_kda_update with dt_bias shaped (HV,) (per-head only), (num_heads, K) with an unfused head count, or any layout whose numel differs from HV*K.

Common situations: Checkpoint stores dt_bias per attention head while the kernel is configured with fused HV = heads*K (or vice versa); a partial state-dict conversion missed dt_bias remapping when porting a KDA model into sglang.

Related errors

• Unexpected A_log shape: {A_log.shape}; expected numel={HV}
• Unexpected initial_state_source shape: {initial_state_source
• Unexpected a shape for varlen: {a.shape}
• Unexpected a shape for dense: {a.shape}
• KDA cutedsl: safe_gate (lower_bound) not yet supported

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/54065cc88a63ac9f. Report an issue: GitHub.