sgl-project/sglang · error · ValueError

cos/sin shape does not cover image tokens and head_dim

Error message

cos/sin shape does not cover image tokens and head_dim

What it means

The RoPE table must cover at least the number of image tokens (cos.shape[0] >= img_tokens) and have exactly head_dim//2 columns. Otherwise the kernel would read out of bounds when applying rotary embeddings to image tokens.

Source

Thrown at python/sglang/kernels/ops/diffusion/rope/hunyuan_qkv_pack_triton.py:183

        raise ValueError("QKV tensors must be CUDA bfloat16 tensors")
    if any(x.device != img_q.device for x in tensors):
        raise ValueError("QKV tensors must be on the same CUDA device")
    batch, img_tokens, num_heads, head_dim = img_q.shape
    txt_tokens = txt_q.shape[1]
    expected_img = (batch, img_tokens, num_heads, head_dim)
    expected_txt = (batch, txt_tokens, num_heads, head_dim)
    if any(tuple(x.shape) != expected_img for x in (img_q, img_k, img_v)):
        raise ValueError("image QKV shapes must match")
    if any(tuple(x.shape) != expected_txt for x in (txt_q, txt_k, txt_v)):
        raise ValueError("text QKV shapes must match")
    if any(x.stride(-1) != 1 for x in tensors):
        raise ValueError("QKV last dimensions must be contiguous")
    if head_dim <= 0 or head_dim > 128 or head_dim % 2:
        raise ValueError("head_dim must be positive, even, and <= 128")
    if cos.ndim != 2 or sin.ndim != 2 or cos.shape != sin.shape:
        raise ValueError("cos and sin must have matching [S, D/2] shapes")
    if cos.shape[0] < img_tokens or cos.shape[1] != head_dim // 2:
        raise ValueError("cos/sin shape does not cover image tokens and head_dim")
    if not cos.is_cuda or not sin.is_cuda or cos.stride(-1) != 1 or sin.stride(-1) != 1:
        raise ValueError("cos and sin must be CUDA and last-dim contiguous")
    if cos.device != img_q.device or sin.device != img_q.device:
        raise ValueError("QKV and cos/sin tensors must be on the same CUDA device")

    total_tokens = img_tokens + txt_tokens
    storage = torch.empty(
        (3, batch, total_tokens, num_heads, head_dim),
        device=img_q.device,
        dtype=img_q.dtype,
    )
    args = []
    for x in tensors:
        args.extend((x.stride(0), x.stride(1), x.stride(2)))
    with torch.cuda.device(img_q.device):
        _hunyuan_qkv_rope_pack_kernel[
            lambda meta: (
                batch * total_tokens,

View on GitHub (pinned to 0132848349)

Solutions

1. Precompute cos/sin with rows >= max image token count.
2. Use head_dim//2 columns (half-split layout).
3. Check that position ids for image tokens stay within the table.

Example fix

// before
cos = torch.randn(txt_tokens, head_dim)  # wrong rows AND columns
// after
cos = torch.randn(max(img_tokens, txt_tokens), head_dim // 2, device='cuda')
sin = torch.randn_like(cos)

assert cos.shape[0] >= img_tokens and cos.shape[1] == head_dim // 2

When it happens

Trigger: Passing a cos/sin table sized for text-only sequence lengths, or with columns for full head_dim rather than head_dim/2, while img tokens exceed the table's rows.

Common situations: Mixing text RoPE tables with image-token inputs in multimodal Hunyuan pipelines, or off-by-one max-position configurations.

Related errors

• cos and sin must have matching [S, D/2] shapes
• QwenImage RoPE text cache overflow before denoising: require
• Fused QK-Norm + RoPE kernel only supports float16/bfloat16,
• txt_freqs_cis must be a 2D cos_sin_cache tensor
• Qwen-VL position_ids do not match the attention input

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1d2e7667c4d355fb. Report an issue: GitHub.