{"record":{"id":"555b395489c9c097","repo":"sgl-project/sglang","slug":"invalid-v-out-shape-for-fused-kv-materialization","errorCode":null,"errorMessage":"Invalid v_out shape for fused KV materialization: got {tuple(v_out.shape)}, expected {expected_shape}.","messagePattern":"Invalid v_out shape for fused KV materialization: got (.+?), expected (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":201,"sourceCode":"    if k_out is None:\n        k_out = torch.empty(expected_shape, dtype=kv.dtype, device=kv.device)\n    else:\n        if k_out.shape != expected_shape:\n            raise ValueError(\n                \"Invalid k_out shape for fused KV materialization: \"\n                f\"got {tuple(k_out.shape)}, expected {expected_shape}.\"\n            )\n        if k_out.device != kv.device or k_out.dtype != kv.dtype:\n            raise ValueError(\n                \"Invalid k_out device/dtype for fused KV materialization: \"\n                f\"got device={k_out.device}, dtype={k_out.dtype}, \"\n                f\"expected device={kv.device}, dtype={kv.dtype}.\"\n            )\n    if v_out is None:\n        v_out = torch.empty_like(k_out)\n    else:\n        if v_out.shape != expected_shape:\n            raise ValueError(\n                \"Invalid v_out shape for fused KV materialization: \"\n                f\"got {tuple(v_out.shape)}, expected {expected_shape}.\"\n            )\n        if v_out.device != kv.device or v_out.dtype != kv.dtype:\n            raise ValueError(\n                \"Invalid v_out device/dtype for fused KV materialization: \"\n                f\"got device={v_out.device}, dtype={v_out.dtype}, \"\n                f\"expected device={kv.device}, dtype={kv.dtype}.\"\n            )\n\n    _fused_norm_rope_kernel_stacked[(total_ctx, num_kv_heads, n_layers)](\n        kv,\n        k_norm_weight,\n        eps,\n        cos_sin_cache,\n        positions,\n        k_out,\n        v_out,","sourceCodeStart":183,"sourceCodeEnd":219,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L183-L219","documentation":"A caller-supplied v_out buffer must have shape [n_layers, total_ctx, num_kv_heads, head_dim], identical to k_out's required layout.","triggerScenarios":"Passing v_out sized or ordered differently from k_out (e.g. only [total_ctx, ...] or sized for a shorter context).","commonSituations":"Reusing stale cache buffers sized for a previous batch, or mirroring an incorrect k_out allocation into v_out.","solutions":["Allocate v_out identically to k_out (torch.empty_like(k_out)) or pass None.","Verify total_ctx and n_layers match the kv input."],"exampleFix":"// before\nv_out = torch.empty(total_ctx, H, D)\n// after\nv_out = None  # or k_out.new_empty((n_layers, total_ctx, H, D))","handlingStrategy":"validation","validationCode":"expected = (n_layers, total_ctx, num_kv_heads, head_dim)\nassert v_out is None or v_out.shape == expected","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Allocate v_out with torch.empty_like(k_out) or pass None."],"tags":["shape-validation","kv-cache"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}