{"record":{"id":"8654af48e8f46af3","repo":"sgl-project/sglang","slug":"dt-bias-must-have-hv-k-elements-got-dt-bia","errorCode":null,"errorMessage":"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()}).","messagePattern":"`dt_bias` must have (.+?) elements \\(got (.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/fla/fused_recurrent.py","lineNumber":599,"sourceCode":"        raise ValueError(\n            f\"`initial_state` must be a 4D tensor (got ndim={initial_state.ndim}).\"\n        )\n    if initial_state.stride(-1) != 1:\n        raise ValueError(\"`initial_state` must be contiguous in the last dim.\")\n    HV, V, K = initial_state.shape[-3:]\n    if a.shape[1] != HV * K:\n        raise ValueError(\n            f\"`a` must have shape [B, HV*K] with HV={HV}, K={K} \"\n            f\"(got a.shape={tuple(a.shape)}).\"\n        )\n    if b.shape[1] != HV:\n        raise ValueError(\n            f\"`b` must have shape [B, HV] with HV={HV} (got b.shape={tuple(b.shape)}).\"\n        )\n    if A_log.numel() != HV:\n        raise ValueError(f\"`A_log` must have {HV} elements (got {A_log.numel()}).\")\n    if dt_bias.numel() != HV * K:\n        raise ValueError(\n            f\"`dt_bias` must have {HV * K} elements (got {dt_bias.numel()}).\"\n        )\n    if out.shape != (B, 1, HV, V):\n        raise ValueError(\n            f\"`out` must have shape {(B, 1, HV, V)} (got out.shape={tuple(out.shape)}).\"\n        )\n\n    qkv_dim = mixed_qkv.shape[1]\n    qk_dim = qkv_dim - HV * V\n    if qk_dim <= 0 or qk_dim % 2 != 0:\n        raise ValueError(\n            f\"Invalid packed `mixed_qkv` last dim={qkv_dim} for HV={HV}, V={V}.\"\n        )\n    q_dim = qk_dim // 2\n    if q_dim % K != 0:\n        raise ValueError(\n            f\"Invalid packed Q size {q_dim}: must be divisible by K={K}. \"\n            \"KDA packed decode requires num_q_heads == num_k_heads and \"","sourceCodeStart":581,"sourceCodeEnd":617,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/fla/fused_recurrent.py#L581-L617","documentation":"The KDA packed decode kernel requires the dt_bias tensor to contain exactly HV*K elements, one delta-time bias per (head, key-dim) pair. This is a shape contract check inside fused_recurrent_kda_packed_decode before launching the Triton kernel. A mismatch means the model's dt bias projection does not line up with the configured head/value geometry.","triggerScenarios":"Calling fused_recurrent_kda_packed_decode with a dt_bias tensor whose numel() differs from HV*K, e.g. passing a per-head bias (HV elements) instead of per-(head, key) bias, or using HV/V/K values inconsistent with the checkpoint.","commonSituations":"Porting a KDA (Kimi Delta Attention) checkpoint whose head_dim or num_key_heads differs from the config used to build dt_bias; slicing dt_bias incorrectly when unpacking a fused projection; mismatched num_v_heads vs num_k_heads settings.","solutions":["Verify dt_bias.numel() == HV * K (e.g. num_v_heads * head_v_dim * head_k_dim) and reshape/reproject the checkpoint's dt bias to that size","Double-check the HV and K values you derive from mixed_qkv/config; they must match the checkpoint geometry","If the model genuinely has a smaller dt bias, expand/repeat it per key dim before calling the kernel"],"exampleFix":"// before\ndt_bias = ckpt['dt_bias']  # [HV]\nout = fused_recurrent_kda_packed_decode(..., dt_bias=dt_bias, ...)\n// after\nassert dt_bias.numel() == HV * K, (dt_bias.shape, HV, K)\ndt_bias = dt_bias.repeat_interleave(K)  # [HV*K] if checkpoint stores per-head\nout = fused_recurrent_kda_packed_decode(..., dt_bias=dt_bias, ...)","handlingStrategy":"validation","validationCode":"HV = num_v_heads * head_v_dim\nassert dt_bias.numel() == HV * K, f'dt_bias {dt_bias.numel()} != HV*K {HV*K}'","typeGuard":"def valid_kda_dt_bias(dt_bias: torch.Tensor, HV: int, K: int) -> bool:\n    return dt_bias.numel() == HV * K","tryCatchPattern":null,"preventionTips":["Validate all KDA geometry (HV, K, V) against the checkpoint before the decode call","Centralize shape derivation from mixed_qkv in one helper to keep dims consistent"],"tags":["pytorch","tensor-shape","kda","linear-attention","validation"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}