sgl-project/sglang · error · ValueError

vis_freqs_cis is required for fused QK-Norm + RoPE kernel

Error message

vis_freqs_cis is required for fused QK-Norm + RoPE kernel

What it means

Joy image DiT's single-stream block uses a fused QK-Norm + RoPE kernel for the image branch that requires the precomputed rotary cos_sin cache. If forward is called with vis_freqs_cis=None, this ValueError is raised before the fused kernel invocation.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/joy_image.py:253

            txt_mod1_shift,
            txt_mod1_scale,
            txt_mod1_gate,
            txt_mod2_shift,
            txt_mod2_scale,
            txt_mod2_gate,
        ) = self.txt_mod(vec)

        # Image attention
        img_modulated = self.fused_modulate_img_norm1(
            img, shift=img_mod1_shift, scale=img_mod1_scale
        )
        img_qkv, _ = self.img_attn_qkv(img_modulated)
        img_q, img_k, img_v = rearrange(
            img_qkv, "B L (K H D) -> K B L H D", K=3, H=self.local_heads_num
        )

        if vis_freqs_cis is None:
            raise ValueError(
                "vis_freqs_cis is required for fused QK-Norm + RoPE kernel"
            )
        if not (isinstance(vis_freqs_cis, torch.Tensor) and vis_freqs_cis.dim() == 2):
            raise ValueError("vis_freqs_cis must be a 2D cos_sin_cache tensor")
        if img_q.dtype not in (torch.float16, torch.bfloat16):
            raise ValueError(
                f"Fused QK-Norm + RoPE kernel only supports float16/bfloat16, but got {img_q.dtype}"
            )
        img_q = img_q.contiguous()
        img_k = img_k.contiguous()
        img_q, img_k = apply_qk_norm_with_optional_rope(
            q=img_q,
            k=img_k,
            q_norm=self.img_attn_q_norm,
            k_norm=self.img_attn_k_norm,
            head_dim=img_q.shape[-1],
            cos_sin_cache=vis_freqs_cis,
            is_neox=False,

View on GitHub (pinned to 0132848349)

Solutions

1. Pass the precomputed 2D vis cos_sin cache (vis_freqs_cis) to forward
2. Build the cache from the model's rotary embedding with the correct max sequence length before the denoising loop
3. Use the pipeline code provided with joy_image which constructs and forwards both freqs_cis tensors

Example fix

# before
out = block(hidden_states, ..., txt_freqs_cis=f, vis_freqs_cis=None)

# after
vis_freqs_cis = rotary_emb.get_freqs_cis(seq_len_img).to(device)
out = block(hidden_states, ..., txt_freqs_cis=f, vis_freqs_cis=vis_freqs_cis)

assert vis_freqs_cis is not None, 'vis_freqs_cis required for fused QK-Norm+RoPE'

def has_vis_freqs(vis_freqs_cis) -> bool:
    return vis_freqs_cis is not None

When it happens

Trigger: Calling the joy_image block forward without the vis_freqs_cis argument — e.g. a custom pipeline that precomputes text freqs but forgets the vision-side cache, or passing None as a placeholder.

Common situations: Writing a standalone test/harness for the block without building the RoPE cache; refactors that thread freqs_cis through kwargs and drop the vis_ one.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• encoder_hidden_states is required when encoder_key_value is
• Didn't get guidance strength for guidance distilled model.
• vis_freqs_cis must be a 2D cos_sin_cache tensor
• QKV tensors must have shape [B, S, H, D]
• Got {axes_dim} but expected positional dim {pe_dim}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b5e8246d5b205451. Report an issue: GitHub.