{"record":{"id":"87a53caf4a0ffd9b","repo":"sgl-project/sglang","slug":"invalid-fused-kv-projection-shape-got-tuple-kv-s","errorCode":null,"errorMessage":"Invalid fused KV projection shape: got {tuple(kv.shape)}, expected trailing dim {kv_size * 2}.","messagePattern":"Invalid fused KV projection shape: got (.+?), expected trailing dim (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":154,"sourceCode":"    v_out: Optional[torch.Tensor] = None,\n) -> tuple[torch.Tensor, torch.Tensor]:\n    \"\"\"Fused RMSNorm + RoPE materialization for all layers.\"\"\"\n    if kv.ndim != 3:\n        raise ValueError(\n            \"Invalid stacked fused KV projection shape: \"\n            f\"got {tuple(kv.shape)}, expected 3D [total_ctx, n_layers, kv_size*2].\"\n        )\n\n    total_ctx, n_layers, kv_dim = kv.shape\n    if total_ctx == 0:\n        empty = torch.empty(\n            (n_layers, 0, num_kv_heads, head_dim), dtype=kv.dtype, device=kv.device\n        )\n        return empty, empty\n\n    kv_size = num_kv_heads * head_dim\n    if kv_dim != kv_size * 2:\n        raise ValueError(\n            \"Invalid fused KV projection shape: \"\n            f\"got {tuple(kv.shape)}, expected trailing dim {kv_size * 2}.\"\n        )\n    if rotary_dim <= 0 or rotary_dim > head_dim or rotary_dim % 2 != 0:\n        raise ValueError(\n            \"Invalid fused KV rotary/head dim pair: \"\n            f\"rotary_dim={rotary_dim}, head_dim={head_dim}.\"\n        )\n    if k_norm_weight.shape != (n_layers, head_dim):\n        raise ValueError(\n            \"Invalid stacked k_norm_weight shape for fused KV materialization: \"\n            f\"got {tuple(k_norm_weight.shape)}, expected {(n_layers, head_dim)}.\"\n        )\n    if eps.shape != (n_layers,):\n        raise ValueError(\n            \"Invalid stacked eps shape for fused KV materialization: \"\n            f\"got {tuple(eps.shape)}, expected {(n_layers,)}.\"\n        )","sourceCodeStart":136,"sourceCodeEnd":172,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L136-L172","documentation":"The trailing dimension of the stacked fused KV tensor must equal exactly 2 * num_kv_heads * head_dim (K and V concatenated per layer). The code derives kv_size from num_kv_heads and head_dim and rejects any other trailing dim.","triggerScenarios":"Passing kv with last dim != num_kv_heads*head_dim*2, or calling with num_kv_heads/head_dim that don't match how the projection weights were fused.","commonSituations":"Model config mismatch: head_dim or num_kv_heads computed differently (e.g. derived from hidden_size/num_attention_heads) than the checkpoint's fused qkv weight layout.","solutions":["Verify num_kv_heads and head_dim match the model config used to build the stacked projections.","Re-check that the fused qkv weight packs exactly K then V with total width 2*kv_size.","Print kv.shape vs expected num_kv_heads*head_dim*2 and reconcile the difference."],"exampleFix":"// before\nmat = FusedKVMaterializer(..., num_kv_heads=8, head_dim=128)  # kv trailing dim 4096\n// after\nmat = FusedKVMaterializer(..., num_kv_heads=8, head_dim=128)  # kv trailing dim must be 8*128*2=2048","handlingStrategy":"validation","validationCode":"expected = num_kv_heads * head_dim * 2\nassert kv.shape[-1] == expected, (kv.shape, expected)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive num_kv_heads/head_dim from the same config used to build the projection weights.","Add a unit test asserting trailing-dim invariants."],"tags":["shape-validation","speculative-decoding","fused-kernel"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}