sgl-project/sglang · error · RuntimeError

image_rotary_emb must be cos_sin_cache tensors

Error message

image_rotary_emb must be cos_sin_cache tensors

What it means

The attention forward expects image_rotary_emb to be a (cos_sin_cache_img, cos_sin_cache_txt) pair of 2-D tensors (the precomputed cache format used with fused RoPE kernels). Passing tuple/list of per-position cos/sin tensors (diffusers-style 4-tuples) raises RuntimeError.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/qwen_image.py:715

            txt_value,
        ) = _get_qkv_projections(self, hidden_states, encoder_hidden_states)

        # Reshape for multi-head attention
        img_query = img_query.unflatten(-1, (self.local_num_heads, self.head_dim))
        img_key = img_key.unflatten(-1, (self.local_num_heads, self.head_dim))
        img_value = img_value.unflatten(-1, (self.local_num_heads, self.head_dim))

        txt_query = txt_query.unflatten(-1, (self.local_num_heads, self.head_dim))
        txt_key = txt_key.unflatten(-1, (self.local_num_heads, self.head_dim))
        txt_value = txt_value.unflatten(-1, (self.local_num_heads, self.head_dim))

        img_cache = txt_cache = None
        if image_rotary_emb is not None:
            if not (
                isinstance(image_rotary_emb[0], torch.Tensor)
                and image_rotary_emb[0].dim() == 2
            ):
                raise RuntimeError("image_rotary_emb must be cos_sin_cache tensors")

            img_cache, txt_cache = image_rotary_emb

        if self.qk_norm:
            img_query, img_key = apply_qk_norm_with_optional_rope(
                q=img_query,
                k=img_key,
                q_norm=self.norm_q,
                k_norm=self.norm_k,
                head_dim=self.head_dim,
                cos_sin_cache=img_cache,
                is_neox=False,
                allow_inplace=True,
            )
            txt_query, txt_key = apply_qk_norm_with_optional_rope(
                q=txt_query,
                k=txt_key,
                q_norm=self.norm_added_q,

View on GitHub (pinned to 0132848349)

Solutions

1. Convert to cos_sin_cache format via the model's rope cache helper (e.g. precompute/stack cos,sin into a 2-D cache per stream)
2. Pass the caches produced by the model's own rotary utility rather than external cos/sin lists

Example fix

# before
img_cache, txt_cache = image_rotary_emb  # diffusers 4-tuple -> RuntimeError
# after
rope_caches = build_cos_sin_cache(head_dim, max_seq, device, dtype)
img_cache, txt_cache = rope_caches.img, rope_caches.txt

img_c, txt_c = image_rotary_emb
assert torch.is_tensor(img_c) and img_c.dim() == 2 and torch.is_tensor(txt_c)

def is_cos_sin_cache(re) -> bool:
    return (
        isinstance(re, (tuple, list)) and len(re) == 2
        and all(torch.is_tensor(t) and t.dim() == 2 for t in re)
    )

When it happens

Trigger: Passing diffusers-style rotary outputs (cos, sin lists of [B,S,D] tensors) instead of the 2-D cos_sin_cache tensors this implementation consumes.

Common situations: Porting pipelines from diffusers Qwen-Image; rope utility returning a different format after a refactor.

Understand the failure class

Background: Invalid argument type errors: "must be of type string", "expected X, got Y", and ERR_INVALID_ARG_TYPE explained — this error's family across 15 libraries.

Related errors

• QwenImage RoPE text cache overflow before denoising: require
• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D
• rope_pool_fused expects pool tensors to be 3-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7e09eca68500e01d. Report an issue: GitHub.