{"record":{"id":"a9963382f08165b8","repo":"sgl-project/sglang","slug":"rope-config-mismatch-across-layers-for-fused-kv-pa","errorCode":null,"errorMessage":"RoPE config mismatch across layers for fused KV path: expected (rotary_dim={self.rotary_dim}, neox={self.is_neox_style}), got (rotary_dim={layer_rotary_dim}, neox={layer_is_neox}) at layer {layer_id}.","messagePattern":"RoPE config mismatch across layers for fused KV path: expected \\(rotary_dim=(.+?), neox=(.+?)\\), got \\(rotary_dim=(.+?), neox=(.+?)\\) at layer (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":316,"sourceCode":"            if int(attn.num_kv_heads) != self.num_kv_heads:\n                raise ValueError(\n                    \"num_kv_heads mismatch across layers for fused KV path: \"\n                    f\"expected {self.num_kv_heads}, got {int(attn.num_kv_heads)} at layer {layer_id}.\"\n                )\n            if int(attn.head_dim) != self.head_dim:\n                raise ValueError(\n                    \"head_dim mismatch across layers for fused KV path: \"\n                    f\"expected {self.head_dim}, got {int(attn.head_dim)} at layer {layer_id}.\"\n                )\n            layer_rotary_dim = int(\n                getattr(attn.rotary_emb, \"rotary_dim\", self.head_dim)\n            )\n            layer_is_neox = bool(getattr(attn.rotary_emb, \"is_neox_style\", True))\n            if (\n                layer_rotary_dim != self.rotary_dim\n                or layer_is_neox != self.is_neox_style\n            ):\n                raise ValueError(\n                    \"RoPE config mismatch across layers for fused KV path: \"\n                    f\"expected (rotary_dim={self.rotary_dim}, neox={self.is_neox_style}), \"\n                    f\"got (rotary_dim={layer_rotary_dim}, neox={layer_is_neox}) at layer {layer_id}.\"\n                )\n\n            qkv_w = attn.qkv_proj.weight\n            kv_weight = qkv_w[attn.q_size : attn.q_size + 2 * attn.kv_size]\n            kv_weights.append(kv_weight)\n            k_norm_weights.append(attn.k_norm.weight)\n            eps_values.append(float(attn.k_norm.variance_epsilon))\n\n        flat_kv_weight = torch.stack(kv_weights).reshape(\n            self.n_layers * self.layer_out_dim, -1\n        )\n        self.flat_kv_weight_t = flat_kv_weight.transpose(0, 1).contiguous()\n        self.k_norm_weights = torch.stack(k_norm_weights).contiguous()\n        self.eps_values = torch.tensor(\n            eps_values, dtype=torch.float32, device=self.device","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L298-L334","documentation":"Every layer's rotary_emb must have the same rotary_dim and is_neox_style as the reference config captured in __init__, because one shared cos/sin cache and kernel layout serve all layers.","triggerScenarios":"A model where one layer's rotary_emb has a different rotary_dim or neox flag (hybrid RoPE configs, partial-rotary on only some layers).","commonSituations":"Checkpoints with mixed full/partial rotary (e.g. some search-optimized variants), or custom per-layer rotary_emb construction with inconsistent attributes.","solutions":["Make rotary configs uniform across layers or disable the fused KV path.","Check getattr defaults: layers whose rotary_emb lacks rotary_dim default to head_dim — set the attribute explicitly if partial.","Diff each layer's rotary_emb.rotary_dim/is_neox_style during model init to catch divergence early."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"cfgs = {(int(getattr(l.self_attn.rotary_emb,'rotary_dim',hd)), bool(getattr(l.self_attn.rotary_emb,'is_neox_style',True))) for l in layers}\nassert len(cfgs) == 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Set rotary_dim explicitly on every layer's rotary_emb when partial rotary is used."],"tags":["config-validation","rope"],"backgroundTag":"config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}