{"record":{"id":"c2621b0c3518cdba","repo":"sgl-project/sglang","slug":"invalid-fused-kv-rotary-head-dim-pair-rotary-dim-c2621b","errorCode":null,"errorMessage":"Invalid fused KV rotary/head dim pair: rotary_dim={self.rotary_dim}, head_dim={self.head_dim}.","messagePattern":"Invalid fused KV rotary/head dim pair: rotary_dim=(.+?), head_dim=(.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/speculative/fused_kv_materialize.py","lineNumber":272,"sourceCode":"        head_dim: int,\n        device: torch.device,\n        max_position_hint: Optional[int] = None,\n    ):\n        self.num_kv_heads = num_kv_heads\n        self.head_dim = head_dim\n        self.rotary_emb = rotary_emb\n        self.n_layers = len(layers)\n        self.device = device\n        self.kv_size = self.num_kv_heads * self.head_dim\n        self.layer_out_dim = 2 * self.kv_size\n\n        self.rotary_dim = int(getattr(rotary_emb, \"rotary_dim\", head_dim))\n        self.is_neox_style = bool(getattr(rotary_emb, \"is_neox_style\", True))\n\n        if not self.is_neox_style:\n            raise NotImplementedError(\"Only neox-style RoPE is supported.\")\n        if self.rotary_dim <= 0 or self.rotary_dim > self.head_dim:\n            raise ValueError(\n                \"Invalid fused KV rotary/head dim pair: \"\n                f\"rotary_dim={self.rotary_dim}, head_dim={self.head_dim}.\"\n            )\n\n        self.max_position_hint = (\n            max(int(max_position_hint) - 1, 0)\n            if max_position_hint is not None\n            else None\n        )\n        self._reserved_rope_cache_len = int(\n            getattr(self.rotary_emb, \"cos_sin_cache\", torch.empty((0,))).shape[0]\n        )\n        self._mm_out_supported = True\n        self._workspace_capacity = 0\n        self._workspace_dtype: Optional[torch.dtype] = None\n        self._proj_workspace: Optional[torch.Tensor] = None\n        self._k_workspace: Optional[torch.Tensor] = None\n        self._v_workspace: Optional[torch.Tensor] = None","sourceCodeStart":254,"sourceCodeEnd":290,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/speculative/fused_kv_materialize.py#L254-L290","documentation":"In the materializer __init__, rotary_dim (read from rotary_emb.rotary_dim, defaulting to head_dim) must satisfy 0 < rotary_dim <= head_dim. Values outside this range can't index the cos/sin cache correctly.","triggerScenarios":"A rotary_emb whose rotary_dim is 0, negative, or exceeds head_dim (misconfigured custom rotary module or corrupted config).","commonSituations":"Custom RotaryEmbedding subclass that doesn't set rotary_dim (getattr default is head_dim, so this fires only when explicitly wrong), or hand-edited model configs.","solutions":["Ensure rotary_emb.rotary_dim is set to the model's true partial-rotary dim, or omit it so head_dim is used.","Validate config.rope parameters against the checkpoint before constructing the materializer."],"exampleFix":"// before\nrotary_emb.rotary_dim = 0\n// after\nrotary_emb.rotary_dim = head_dim  # or delete attr to use default","handlingStrategy":"validation","validationCode":"rd = int(getattr(rotary_emb, 'rotary_dim', head_dim))\nassert 0 < rd <= head_dim","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Sanity-check rotary config against the checkpoint at load time."],"tags":["config-validation","rope"],"backgroundTag":"invalid-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}