sgl-project/sglang · error · ValueError

txt_freqs_cis must be a 2D cos_sin_cache tensor

Error message

txt_freqs_cis must be a 2D cos_sin_cache tensor

What it means

JoyImage validates that the text-branch rotary embedding cache is a 2D cos_sin tensor of shape (seq_len, rot_dim*2). Anything else (3D freqs_cis tuple, a list, a wrongly shaped tensor) breaks the fused QK-Norm+RoPE call which indexes a flat 2D cache.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/joy_image.py:288

            cos_sin_cache=vis_freqs_cis,
            is_neox=False,
            allow_inplace=True,
        )
        img_q, img_k = img_q.to(img_v), img_k.to(img_v)

        # Text attention
        txt_modulated = self.fused_modulate_txt_norm1(
            txt, shift=txt_mod1_shift, scale=txt_mod1_scale
        )
        txt_qkv, _ = self.txt_attn_qkv(txt_modulated)
        txt_q, txt_k, txt_v = rearrange(
            txt_qkv, "B L (K H D) -> K B L H D", K=3, H=self.local_heads_num
        )

        if txt_freqs_cis is not None and not (
            isinstance(txt_freqs_cis, torch.Tensor) and txt_freqs_cis.dim() == 2
        ):
            raise ValueError("txt_freqs_cis must be a 2D cos_sin_cache tensor")
        txt_q = txt_q.contiguous()
        txt_k = txt_k.contiguous()
        txt_q, txt_k = apply_qk_norm_with_optional_rope(
            q=txt_q,
            k=txt_k,
            q_norm=self.txt_attn_q_norm,
            k_norm=self.txt_attn_k_norm,
            head_dim=txt_q.shape[-1],
            cos_sin_cache=txt_freqs_cis,
            is_neox=False,
            allow_inplace=True,
        )
        txt_q, txt_k = txt_q.to(txt_v), txt_k.to(txt_v)

        # Attention
        joint_query = torch.cat([img_q, txt_q], dim=1)
        joint_key = torch.cat([img_k, txt_k], dim=1)
        joint_value = torch.cat([img_v, txt_v], dim=1)

View on GitHub (pinned to 0132848349)

Solutions

1. Pass a 2D concatenated cos_sin cache: torch.cat([cos, sin], dim=-1) with shape (max_seq, head_dim)
2. Or pass txt_freqs_cis=None to skip RoPE on the text branch if the model config expects no text RoPE
3. Check the freqs-cache builder in the runtime (it should emit 2D caches) and fix the upstream constructor

Example fix

// before
freqs = compute_freqs(...)  # returns (cos, sin) tuple
dit(x, txt_freqs_cis=freqs)

// after
cos, sin = compute_freqs(...)
cache = torch.cat([cos, sin], dim=-1).to(x.dtype)  # (seq, dim*2)
dit(x, txt_freqs_cis=cache)

def as_cos_sin_cache(t) -> torch.Tensor | None:\n    if t is None:\n        return None\n    if isinstance(t, (tuple, list)):\n        t = torch.cat(list(t), dim=-1)\n    assert isinstance(t, torch.Tensor) and t.dim() == 2, f'want 2D cache, got {type(t)} {getattr(t, "shape", None)}'\n    return t

def is_2d_cos_sin_cache(t) -> bool:\n    return isinstance(t, torch.Tensor) and t.dim() == 2

When it happens

Trigger: Passing txt_freqs_cis as the tuple returned by classic rope precomputation, as a (seq, head, dim) tensor, or as a non-tensor, while txt_freqs_cis is not None.

Common situations: Porting code from a model whose RoPE API takes (cos, sin) tuples; passing vision-branch freqs format to the text branch; older checkpoints/configs producing a 3D cache.

Related errors

• Fused QK-Norm + RoPE kernel only supports float16/bfloat16,
• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D
• rope_pool_fused expects pool tensors to be 3-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/369dc2cdf3ad74e8. Report an issue: GitHub.