{"record":{"id":"42bfe61319f513c9","repo":"sgl-project/sglang","slug":"invalid-fused-kv-rotary-head-dim-pair-rotary-dim","errorCode":null,"errorMessage":"Invalid fused KV rotary/head dim pair: rotary_dim={rotary_dim}, head_dim={head_dim}.","messagePattern":"Invalid fused KV rotary/head dim pair: rotary_dim=(.+?), head_dim=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":159,"sourceCode":"            \"Invalid stacked fused KV projection shape: \"\n            f\"got {tuple(kv.shape)}, expected 3D [total_ctx, n_layers, kv_size*2].\"\n        )\n\n    total_ctx, n_layers, kv_dim = kv.shape\n    if total_ctx == 0:\n        empty = torch.empty(\n            (n_layers, 0, num_kv_heads, head_dim), dtype=kv.dtype, device=kv.device\n        )\n        return empty, empty\n\n    kv_size = num_kv_heads * head_dim\n    if kv_dim != kv_size * 2:\n        raise ValueError(\n            \"Invalid fused KV projection shape: \"\n            f\"got {tuple(kv.shape)}, expected trailing dim {kv_size * 2}.\"\n        )\n    if rotary_dim <= 0 or rotary_dim > head_dim or rotary_dim % 2 != 0:\n        raise ValueError(\n            \"Invalid fused KV rotary/head dim pair: \"\n            f\"rotary_dim={rotary_dim}, head_dim={head_dim}.\"\n        )\n    if k_norm_weight.shape != (n_layers, head_dim):\n        raise ValueError(\n            \"Invalid stacked k_norm_weight shape for fused KV materialization: \"\n            f\"got {tuple(k_norm_weight.shape)}, expected {(n_layers, head_dim)}.\"\n        )\n    if eps.shape != (n_layers,):\n        raise ValueError(\n            \"Invalid stacked eps shape for fused KV materialization: \"\n            f\"got {tuple(eps.shape)}, expected {(n_layers,)}.\"\n        )\n\n    half_rotary_dim = rotary_dim // 2\n    BLOCK_HD = triton.next_power_of_2(head_dim)\n\n    if positions.device != kv.device:","sourceCodeStart":141,"sourceCodeEnd":177,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L141-L177","documentation":"The rotary_dim argument must be positive, no greater than head_dim, and even (RoPE pairs dimensions). Any other value is rejected before launching the Triton kernel.","triggerScenarios":"Passing rotary_dim=0, an odd rotary_dim, or a partial-rotary dim larger than head_dim to _fused_norm_rope_stacked / materialize.","commonSituations":"A model with no rotary_emb (rotary_dim defaulting to 0) or a GPT-J-style odd partial rotary config being run through the fused KV path.","solutions":["Pass the model's actual rotary_emb.rotary_dim (defaults to head_dim for full RoPE).","Skip the fused KV path for models without RoPE or with invalid rotary configs.","If rotary_dim < head_dim, ensure it is even (e.g. use rotary_dim - rotary_dim % 2 only if semantically correct)."],"exampleFix":"// before\nmat.materialize(kv, positions, rotary_dim=head_dim + 2)\n// after\nmat.materialize(kv, positions, rotary_dim=head_dim)","handlingStrategy":"validation","validationCode":"assert 0 < rotary_dim <= head_dim and rotary_dim % 2 == 0","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pass rotary_dim straight from the model's rotary_emb rather than computing it ad hoc.","Skip the fused path for models without RoPE."],"tags":["config-validation","rope","speculative-decoding"],"backgroundTag":"invalid-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}