{"record":{"id":"369dc2cdf3ad74e8","repo":"sgl-project/sglang","slug":"txt-freqs-cis-must-be-a-2d-cos-sin-cache-tensor","errorCode":null,"errorMessage":"txt_freqs_cis must be a 2D cos_sin_cache tensor","messagePattern":"txt_freqs_cis must be a 2D cos_sin_cache tensor","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/joy_image.py","lineNumber":288,"sourceCode":"            cos_sin_cache=vis_freqs_cis,\n            is_neox=False,\n            allow_inplace=True,\n        )\n        img_q, img_k = img_q.to(img_v), img_k.to(img_v)\n\n        # Text attention\n        txt_modulated = self.fused_modulate_txt_norm1(\n            txt, shift=txt_mod1_shift, scale=txt_mod1_scale\n        )\n        txt_qkv, _ = self.txt_attn_qkv(txt_modulated)\n        txt_q, txt_k, txt_v = rearrange(\n            txt_qkv, \"B L (K H D) -> K B L H D\", K=3, H=self.local_heads_num\n        )\n\n        if txt_freqs_cis is not None and not (\n            isinstance(txt_freqs_cis, torch.Tensor) and txt_freqs_cis.dim() == 2\n        ):\n            raise ValueError(\"txt_freqs_cis must be a 2D cos_sin_cache tensor\")\n        txt_q = txt_q.contiguous()\n        txt_k = txt_k.contiguous()\n        txt_q, txt_k = apply_qk_norm_with_optional_rope(\n            q=txt_q,\n            k=txt_k,\n            q_norm=self.txt_attn_q_norm,\n            k_norm=self.txt_attn_k_norm,\n            head_dim=txt_q.shape[-1],\n            cos_sin_cache=txt_freqs_cis,\n            is_neox=False,\n            allow_inplace=True,\n        )\n        txt_q, txt_k = txt_q.to(txt_v), txt_k.to(txt_v)\n\n        # Attention\n        joint_query = torch.cat([img_q, txt_q], dim=1)\n        joint_key = torch.cat([img_k, txt_k], dim=1)\n        joint_value = torch.cat([img_v, txt_v], dim=1)","sourceCodeStart":270,"sourceCodeEnd":306,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/joy_image.py#L270-L306","documentation":"JoyImage validates that the text-branch rotary embedding cache is a 2D cos_sin tensor of shape (seq_len, rot_dim*2). Anything else (3D freqs_cis tuple, a list, a wrongly shaped tensor) breaks the fused QK-Norm+RoPE call which indexes a flat 2D cache.","triggerScenarios":"Passing txt_freqs_cis as the tuple returned by classic rope precomputation, as a (seq, head, dim) tensor, or as a non-tensor, while txt_freqs_cis is not None.","commonSituations":"Porting code from a model whose RoPE API takes (cos, sin) tuples; passing vision-branch freqs format to the text branch; older checkpoints/configs producing a 3D cache.","solutions":["Pass a 2D concatenated cos_sin cache: torch.cat([cos, sin], dim=-1) with shape (max_seq, head_dim)","Or pass txt_freqs_cis=None to skip RoPE on the text branch if the model config expects no text RoPE","Check the freqs-cache builder in the runtime (it should emit 2D caches) and fix the upstream constructor"],"exampleFix":"// before\nfreqs = compute_freqs(...)  # returns (cos, sin) tuple\ndit(x, txt_freqs_cis=freqs)\n\n// after\ncos, sin = compute_freqs(...)\ncache = torch.cat([cos, sin], dim=-1).to(x.dtype)  # (seq, dim*2)\ndit(x, txt_freqs_cis=cache)","handlingStrategy":"type-guard","validationCode":"def as_cos_sin_cache(t) -> torch.Tensor | None:\\n    if t is None:\\n        return None\\n    if isinstance(t, (tuple, list)):\\n        t = torch.cat(list(t), dim=-1)\\n    assert isinstance(t, torch.Tensor) and t.dim() == 2, f'want 2D cache, got {type(t)} {getattr(t, \"shape\", None)}'\\n    return t","typeGuard":"def is_2d_cos_sin_cache(t) -> bool:\\n    return isinstance(t, torch.Tensor) and t.dim() == 2","tryCatchPattern":null,"preventionTips":["Standardize all RoPE caches as 2D cat([cos,sin],-1) tensors","Add a shape assert at the boundary of your sampling loop","Never pass (cos, sin) tuples across model APIs"],"tags":["rope","shape-validation","joyimage","multimodal"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}