sgl-project/sglang · error · ValueError

Invalid fused KV rotary/head dim pair: rotary_dim={rotary_di

Error message

Invalid fused KV rotary/head dim pair: rotary_dim={rotary_dim}, head_dim={head_dim}.

What it means

The rotary_dim argument must be positive, no greater than head_dim, and even (RoPE pairs dimensions). Any other value is rejected before launching the Triton kernel.

Source

Thrown at python/sglang/kernels/ops/speculative/fused_kv_materialize.py:159

            "Invalid stacked fused KV projection shape: "
            f"got {tuple(kv.shape)}, expected 3D [total_ctx, n_layers, kv_size*2]."
        )

    total_ctx, n_layers, kv_dim = kv.shape
    if total_ctx == 0:
        empty = torch.empty(
            (n_layers, 0, num_kv_heads, head_dim), dtype=kv.dtype, device=kv.device
        )
        return empty, empty

    kv_size = num_kv_heads * head_dim
    if kv_dim != kv_size * 2:
        raise ValueError(
            "Invalid fused KV projection shape: "
            f"got {tuple(kv.shape)}, expected trailing dim {kv_size * 2}."
        )
    if rotary_dim <= 0 or rotary_dim > head_dim or rotary_dim % 2 != 0:
        raise ValueError(
            "Invalid fused KV rotary/head dim pair: "
            f"rotary_dim={rotary_dim}, head_dim={head_dim}."
        )
    if k_norm_weight.shape != (n_layers, head_dim):
        raise ValueError(
            "Invalid stacked k_norm_weight shape for fused KV materialization: "
            f"got {tuple(k_norm_weight.shape)}, expected {(n_layers, head_dim)}."
        )
    if eps.shape != (n_layers,):
        raise ValueError(
            "Invalid stacked eps shape for fused KV materialization: "
            f"got {tuple(eps.shape)}, expected {(n_layers,)}."
        )

    half_rotary_dim = rotary_dim // 2
    BLOCK_HD = triton.next_power_of_2(head_dim)

    if positions.device != kv.device:

View on GitHub (pinned to 0132848349)

Solutions

1. Pass the model's actual rotary_emb.rotary_dim (defaults to head_dim for full RoPE).
2. Skip the fused KV path for models without RoPE or with invalid rotary configs.
3. If rotary_dim < head_dim, ensure it is even (e.g. use rotary_dim - rotary_dim % 2 only if semantically correct).

Example fix

// before
mat.materialize(kv, positions, rotary_dim=head_dim + 2)
// after
mat.materialize(kv, positions, rotary_dim=head_dim)

assert 0 < rotary_dim <= head_dim and rotary_dim % 2 == 0

When it happens

Trigger: Passing rotary_dim=0, an odd rotary_dim, or a partial-rotary dim larger than head_dim to _fused_norm_rope_stacked / materialize.

Common situations: A model with no rotary_emb (rotary_dim defaulting to 0) or a GPT-J-style odd partial rotary config being run through the fused KV path.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Only neox-style RoPE is supported.
• Invalid fused KV rotary/head dim pair: rotary_dim={self.rota
• num_kv_heads mismatch across layers for fused KV path: expec
• RoPE config mismatch across layers for fused KV path: expect
• {rope_type=} not supported. Choose between 'interleaved' and

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/42bfe61319f513c9. Report an issue: GitHub.