sgl-project/sglang · error · ValueError

Invalid fused KV rotary/head dim pair: rotary_dim={self.rota

Error message

Invalid fused KV rotary/head dim pair: rotary_dim={self.rotary_dim}, head_dim={self.head_dim}.

What it means

In the materializer __init__, rotary_dim (read from rotary_emb.rotary_dim, defaulting to head_dim) must satisfy 0 < rotary_dim <= head_dim. Values outside this range can't index the cos/sin cache correctly.

Source

Thrown at python/sglang/kernels/ops/speculative/fused_kv_materialize.py:272

        head_dim: int,
        device: torch.device,
        max_position_hint: Optional[int] = None,
    ):
        self.num_kv_heads = num_kv_heads
        self.head_dim = head_dim
        self.rotary_emb = rotary_emb
        self.n_layers = len(layers)
        self.device = device
        self.kv_size = self.num_kv_heads * self.head_dim
        self.layer_out_dim = 2 * self.kv_size

        self.rotary_dim = int(getattr(rotary_emb, "rotary_dim", head_dim))
        self.is_neox_style = bool(getattr(rotary_emb, "is_neox_style", True))

        if not self.is_neox_style:
            raise NotImplementedError("Only neox-style RoPE is supported.")
        if self.rotary_dim <= 0 or self.rotary_dim > self.head_dim:
            raise ValueError(
                "Invalid fused KV rotary/head dim pair: "
                f"rotary_dim={self.rotary_dim}, head_dim={self.head_dim}."
            )

        self.max_position_hint = (
            max(int(max_position_hint) - 1, 0)
            if max_position_hint is not None
            else None
        )
        self._reserved_rope_cache_len = int(
            getattr(self.rotary_emb, "cos_sin_cache", torch.empty((0,))).shape[0]
        )
        self._mm_out_supported = True
        self._workspace_capacity = 0
        self._workspace_dtype: Optional[torch.dtype] = None
        self._proj_workspace: Optional[torch.Tensor] = None
        self._k_workspace: Optional[torch.Tensor] = None
        self._v_workspace: Optional[torch.Tensor] = None

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure rotary_emb.rotary_dim is set to the model's true partial-rotary dim, or omit it so head_dim is used.
2. Validate config.rope parameters against the checkpoint before constructing the materializer.

Example fix

// before
rotary_emb.rotary_dim = 0
// after
rotary_emb.rotary_dim = head_dim  # or delete attr to use default

rd = int(getattr(rotary_emb, 'rotary_dim', head_dim))
assert 0 < rd <= head_dim

When it happens

Trigger: A rotary_emb whose rotary_dim is 0, negative, or exceeds head_dim (misconfigured custom rotary module or corrupted config).

Common situations: Custom RotaryEmbedding subclass that doesn't set rotary_dim (getattr default is head_dim, so this fires only when explicitly wrong), or hand-edited model configs.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Invalid fused KV rotary/head dim pair: rotary_dim={rotary_di
• RoPE config mismatch across layers for fused KV path: expect
• {rope_type=} not supported. Choose between 'interleaved' and
• memory_position_mode must be one of {'reference', 'legacy',
• no_rope_layers contains non-binary entries {bad_flags}; each

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c2621b0c3518cdba. Report an issue: GitHub.