{"record":{"id":"e368f8555a9c3801","repo":"sgl-project/sglang","slug":"mismatched-batch-sizes-mixed-qkv-shape-0-b-a-e368f8","errorCode":null,"errorMessage":"Mismatched batch sizes: mixed_qkv.shape[0]={B}, a.shape[0]={a.shape[0]}, b.shape[0]={b.shape[0]}.","messagePattern":"Mismatched batch sizes: mixed_qkv\\.shape\\[0\\]=(.+?), a\\.shape\\[0\\]=(.+?), b\\.shape\\[0\\]=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/helion/kda_decode.py","lineNumber":269,"sourceCode":"\n    device = mixed_qkv.device\n    if any(\n        tensor.device != device\n        for tensor in (\n            a,\n            b,\n            A_log,\n            dt_bias,\n            initial_state,\n            out,\n            ssm_state_indices,\n        )\n    ):\n        raise ValueError(\"All inputs must be on the same device.\")\n\n    B = mixed_qkv.shape[0]\n    if a.shape[0] != B or b.shape[0] != B:\n        raise ValueError(\n            \"Mismatched batch sizes: \"\n            f\"mixed_qkv.shape[0]={B}, a.shape[0]={a.shape[0]}, \"\n            f\"b.shape[0]={b.shape[0]}.\"\n        )\n    if ssm_state_indices.shape[0] != B:\n        raise ValueError(\n            f\"`ssm_state_indices` must have shape [B] \"\n            f\"(got {tuple(ssm_state_indices.shape)}; expected ({B},)).\"\n        )\n\n    if initial_state.ndim != 4:\n        raise ValueError(\n            f\"`initial_state` must be a 4D tensor (got ndim={initial_state.ndim}).\"\n        )\n    if initial_state.stride(-1) != 1:\n        raise ValueError(\"`initial_state` must be contiguous in the last dim.\")\n    HV, V, K = initial_state.shape[-3:]\n    if not _is_power_of_two(K) or not _is_power_of_two(V):","sourceCodeStart":251,"sourceCodeEnd":287,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/helion/kda_decode.py#L251-L287","documentation":"The validator derives batch size B from mixed_qkv.shape[0] and requires a.shape[0] and b.shape[0] to equal it, since each batch row has corresponding gating tensors. A mismatch means inconsistent tensor batches.","triggerScenarios":"Calling packed decode where a or b has fewer/more rows than mixed_qkv (e.g. B=8 qkv with B=4 a).","commonSituations":"Dropped/finished requests removed from one tensor but not others; slicing a/b by a different batch dimension after scheduling.","solutions":["Re-align a and b to the current batch: recompute them from the same request set as mixed_qkv","Check batch filtering logic (finished-request removal) updates all decode tensors consistently"],"exampleFix":"# before\nout = decode(qkv[keep], a, b, ...)  # a,b still full batch\n# after\na, b = a[keep], b[keep]\nout = decode(qkv[keep], a, b, ...)","handlingStrategy":"validation","validationCode":"B = mixed_qkv.shape[0]\nassert a.shape[0] == B and b.shape[0] == B, 'batch mismatch'","typeGuard":"def batches_align(mixed_qkv, a, b) -> bool:\n    B = mixed_qkv.shape[0]\n    return a.shape[0] == B and b.shape[0] == B","tryCatchPattern":null,"preventionTips":["Filter all decode tensors with the same keep-mask atomically","Derive batched tensors from one canonical batch structure"],"tags":["kda","mamba","batch-mismatch","helion"],"backgroundTag":"batch-size-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}