sgl-project/sglang · error · ValueError

vis_freqs_cis must be a 2D cos_sin_cache tensor

Error message

vis_freqs_cis must be a 2D cos_sin_cache tensor

What it means

The fused QK-Norm + RoPE kernel in joy_image takes the rotary cache as a 2D (cos_sin) tensor. If vis_freqs_cis is not a torch.Tensor or does not have exactly 2 dimensions (e.g. a 3D complex cache or a tuple), this ValueError is raised.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/joy_image.py:257

            txt_mod2_scale,
            txt_mod2_gate,
        ) = self.txt_mod(vec)

        # Image attention
        img_modulated = self.fused_modulate_img_norm1(
            img, shift=img_mod1_shift, scale=img_mod1_scale
        )
        img_qkv, _ = self.img_attn_qkv(img_modulated)
        img_q, img_k, img_v = rearrange(
            img_qkv, "B L (K H D) -> K B L H D", K=3, H=self.local_heads_num
        )

        if vis_freqs_cis is None:
            raise ValueError(
                "vis_freqs_cis is required for fused QK-Norm + RoPE kernel"
            )
        if not (isinstance(vis_freqs_cis, torch.Tensor) and vis_freqs_cis.dim() == 2):
            raise ValueError("vis_freqs_cis must be a 2D cos_sin_cache tensor")
        if img_q.dtype not in (torch.float16, torch.bfloat16):
            raise ValueError(
                f"Fused QK-Norm + RoPE kernel only supports float16/bfloat16, but got {img_q.dtype}"
            )
        img_q = img_q.contiguous()
        img_k = img_k.contiguous()
        img_q, img_k = apply_qk_norm_with_optional_rope(
            q=img_q,
            k=img_k,
            q_norm=self.img_attn_q_norm,
            k_norm=self.img_attn_k_norm,
            head_dim=img_q.shape[-1],
            cos_sin_cache=vis_freqs_cis,
            is_neox=False,
            allow_inplace=True,
        )
        img_q, img_k = img_q.to(img_v), img_k.to(img_v)

View on GitHub (pinned to 0132848349)

Solutions

1. Convert the cache to a 2D tensor: cat cos and sin along the feature dim to get [seq, head_dim]
2. Ensure you pass a torch.Tensor, not a tuple or list
3. Build the cache with the model's own rotary-embedding helper that emits the cos_sin 2D layout

Example fix

# before
vis_freqs_cis = torch.view_as_complex(freqs_3d)  # or a (cos, sin) tuple

# after
vis_freqs_cis = torch.cat([cos, sin], dim=-1)  # shape [seq, head_dim], 2D

assert isinstance(vis_freqs_cis, torch.Tensor) and vis_freqs_cis.dim() == 2, vis_freqs_cis.shape if isinstance(vis_freqs_cis, torch.Tensor) else type(vis_freqs_cis)

def is_2d_cos_sin_cache(x) -> bool:
    return isinstance(x, torch.Tensor) and x.dim() == 2

When it happens

Trigger: Passing a freqs_cis in complex/tensor-of-pairs format (3D: [seq, heads, dim]) or a non-tensor (tuple of (cos, sin)) instead of the expected 2D [seq, cos_sin_dim] layout.

Common situations: Reusing a rope cache builder from another model (e.g. flux-style complex freqs) that returns a different shape; passing torch.stack([cos, sin]) without flattening.

Related errors

• Hunyuan3D Paint expects square latents and a matching view c
• vis_freqs_cis is required for fused QK-Norm + RoPE kernel
• QKV tensors must have shape [B, S, H, D]
• {debug_name}: current-chunk rewrite size changed
• encoder_hidden_states is required when encoder_key_value is

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/72032d7005989d9f. Report an issue: GitHub.