{"record":{"id":"0105de096d3a3506","repo":"sgl-project/sglang","slug":"head-dim-mismatch-across-layers-for-fused-kv-path","errorCode":null,"errorMessage":"head_dim mismatch across layers for fused KV path: expected {self.head_dim}, got {int(attn.head_dim)} at layer {layer_id}.","messagePattern":"head_dim mismatch across layers for fused KV path: expected (.+?), got (.+?) at layer (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":304,"sourceCode":"        self._workspace_capacity = 0\n        self._workspace_dtype: Optional[torch.dtype] = None\n        self._proj_workspace: Optional[torch.Tensor] = None\n        self._k_workspace: Optional[torch.Tensor] = None\n        self._v_workspace: Optional[torch.Tensor] = None\n\n        kv_weights = []\n        k_norm_weights = []\n        eps_values = []\n\n        for layer_id, layer in enumerate(layers):\n            attn = layer.self_attn\n            if int(attn.num_kv_heads) != self.num_kv_heads:\n                raise ValueError(\n                    \"num_kv_heads mismatch across layers for fused KV path: \"\n                    f\"expected {self.num_kv_heads}, got {int(attn.num_kv_heads)} at layer {layer_id}.\"\n                )\n            if int(attn.head_dim) != self.head_dim:\n                raise ValueError(\n                    \"head_dim mismatch across layers for fused KV path: \"\n                    f\"expected {self.head_dim}, got {int(attn.head_dim)} at layer {layer_id}.\"\n                )\n            layer_rotary_dim = int(\n                getattr(attn.rotary_emb, \"rotary_dim\", self.head_dim)\n            )\n            layer_is_neox = bool(getattr(attn.rotary_emb, \"is_neox_style\", True))\n            if (\n                layer_rotary_dim != self.rotary_dim\n                or layer_is_neox != self.is_neox_style\n            ):\n                raise ValueError(\n                    \"RoPE config mismatch across layers for fused KV path: \"\n                    f\"expected (rotary_dim={self.rotary_dim}, neox={self.is_neox_style}), \"\n                    f\"got (rotary_dim={layer_rotary_dim}, neox={layer_is_neox}) at layer {layer_id}.\"\n                )\n\n            qkv_w = attn.qkv_proj.weight","sourceCodeStart":286,"sourceCodeEnd":322,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L286-L322","documentation":"All layers must share the same head_dim for the fused KV path — the kernel's BLOCK_HD is derived from a single head_dim. Layer-level head_dim differing from the first layer's raises this.","triggerScenarios":"Models with mixed head dimensions across layers (some new hybrid architectures, misloaded per-layer configs).","commonSituations":"Per-layer attention config lists where head_dim entries differ, or a partially converted checkpoint.","solutions":["Ensure uniform head_dim across all layers before enabling fused KV.","Re-check the model's per-layer config JSON / weights.","Use the fallback per-layer path for mixed head_dim models."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"dims = {int(l.self_attn.head_dim) for l in layers}\nassert len(dims) == 1","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Diff per-layer configs when loading hybrid checkpoints."],"tags":["config-validation","attention"],"backgroundTag":"config-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}