{"record":{"id":"54065cc88a63ac9f","repo":"sgl-project/sglang","slug":"unexpected-dt-bias-shape-dt-bias-shape-expecte","errorCode":null,"errorMessage":"Unexpected dt_bias shape: {dt_bias.shape}; expected numel={HV * K}","messagePattern":"Unexpected dt_bias shape: (.+?); expected numel=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/cutedsl_kda.py","lineNumber":1353,"sourceCode":"    _compiled_kernels[key] = compiled_kernel\n    logger.info(\n        \"CuTe DSL KDA kernel compiled: \"\n        f\"N={N}, H={H}, HV={HV}, K={K}, V={V}, pool_size={pool_size}, \"\n        f\"pool_strides={tuple(h0_source.stride())}, \"\n        f\"small_batch={use_small_batch}, varlen={is_varlen_decode}\"\n    )\n    return compiled_kernel\n\n\ndef _normalize_A_log(A_log: torch.Tensor, HV: int) -> torch.Tensor:\n    if A_log.numel() != HV:\n        raise ValueError(f\"Unexpected A_log shape: {A_log.shape}; expected numel={HV}\")\n    return A_log.reshape(HV).contiguous()\n\n\ndef _normalize_dt_bias(dt_bias: torch.Tensor, HV: int, K: int) -> torch.Tensor:\n    if dt_bias.numel() != HV * K:\n        raise ValueError(\n            f\"Unexpected dt_bias shape: {dt_bias.shape}; expected numel={HV * K}\"\n        )\n    return dt_bias.reshape(HV, K).contiguous()\n\n\ndef _normalize_kda_a(a, *, is_varlen_decode, N, HV, K):\n    \"\"\"Normalize `a` to match the compile-time shape expected by the kernel.\n\n    varlen kernel compiled shape: (N, HV, K)  -- 3D\n    dense kernel compiled shape:  (N, 1, HV, K) -- 4D\n    \"\"\"\n    if is_varlen_decode:\n        # Target: (N, HV, K) -- 3D\n        if a.dim() == 2 and a.shape == (N, HV * K):\n            return a.view(N, HV, K)\n        if a.dim() == 3 and a.shape == (N, HV, K):\n            return a  # already correct\n        if a.dim() == 4 and a.shape == (1, N, HV, K):","sourceCodeStart":1335,"sourceCodeEnd":1371,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/cutedsl_kda.py#L1335-L1371","documentation":"_normalize_dt_bias requires the dt (delta-time) bias to have exactly HV*K elements so it can be reshaped to (HV, K). A different element count raises ValueError listing the expected numel.","triggerScenarios":"Calling cutedsl_fused_sigmoid_gating_kda_update with dt_bias shaped (HV,) (per-head only), (num_heads, K) with an unfused head count, or any layout whose numel differs from HV*K.","commonSituations":"Checkpoint stores dt_bias per attention head while the kernel is configured with fused HV = heads*K (or vice versa); a partial state-dict conversion missed dt_bias remapping when porting a KDA model into sglang.","solutions":["Align HV/K arguments with the checkpoint: HV*K must equal dt_bias.numel()","Reshape dt_bias to (HV, K) once at load time so the guard passes trivially","Add a load-time assertion: assert dt_bias.numel() == HV * K"],"exampleFix":"# before: dt_bias shape (num_heads,) but HV*K expected -> ValueError\n# after\ndt_bias = checkpoint_dt_bias.reshape(HV, K).contiguous()\nupdate = cutedsl_fused_sigmoid_gating_kda_update(A_log, dt_bias, q, k, v, ...)","handlingStrategy":"validation","validationCode":"assert dt_bias.numel() == HV * K, f'dt_bias numel {dt_bias.numel()} != HV*K {HV*K}'","typeGuard":"def is_valid_dt_bias(dt_bias, HV: int, K: int) -> bool:\n    return dt_bias.numel() == HV * K","tryCatchPattern":null,"preventionTips":["Reshape dt_bias to (HV, K) in the checkpoint loader","Add shape assertions in the model's post_load hook"],"tags":["kda","linear-attention","shape-validation","cutedsl"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}