sgl-project/sglang · error · ValueError

cos and sin must have matching [S, D/2] shapes

Error message

cos and sin must have matching [S, D/2] shapes

What it means

The kernel requires cos and sin RoPE tables to both be 2-D with identical shapes of form [S, head_dim/2]. Mismatched or wrong-rank tables indicate a RoPE frequency table that doesn't match the packed layout the kernel expects.

Source

Thrown at python/sglang/kernels/ops/diffusion/rope/hunyuan_qkv_pack_triton.py:181

        raise ValueError("QKV tensors must have shape [B, S, H, D]")
    if any(not x.is_cuda or x.dtype != torch.bfloat16 for x in tensors):
        raise ValueError("QKV tensors must be CUDA bfloat16 tensors")
    if any(x.device != img_q.device for x in tensors):
        raise ValueError("QKV tensors must be on the same CUDA device")
    batch, img_tokens, num_heads, head_dim = img_q.shape
    txt_tokens = txt_q.shape[1]
    expected_img = (batch, img_tokens, num_heads, head_dim)
    expected_txt = (batch, txt_tokens, num_heads, head_dim)
    if any(tuple(x.shape) != expected_img for x in (img_q, img_k, img_v)):
        raise ValueError("image QKV shapes must match")
    if any(tuple(x.shape) != expected_txt for x in (txt_q, txt_k, txt_v)):
        raise ValueError("text QKV shapes must match")
    if any(x.stride(-1) != 1 for x in tensors):
        raise ValueError("QKV last dimensions must be contiguous")
    if head_dim <= 0 or head_dim > 128 or head_dim % 2:
        raise ValueError("head_dim must be positive, even, and <= 128")
    if cos.ndim != 2 or sin.ndim != 2 or cos.shape != sin.shape:
        raise ValueError("cos and sin must have matching [S, D/2] shapes")
    if cos.shape[0] < img_tokens or cos.shape[1] != head_dim // 2:
        raise ValueError("cos/sin shape does not cover image tokens and head_dim")
    if not cos.is_cuda or not sin.is_cuda or cos.stride(-1) != 1 or sin.stride(-1) != 1:
        raise ValueError("cos and sin must be CUDA and last-dim contiguous")
    if cos.device != img_q.device or sin.device != img_q.device:
        raise ValueError("QKV and cos/sin tensors must be on the same CUDA device")

    total_tokens = img_tokens + txt_tokens
    storage = torch.empty(
        (3, batch, total_tokens, num_heads, head_dim),
        device=img_q.device,
        dtype=img_q.dtype,
    )
    args = []
    for x in tensors:
        args.extend((x.stride(0), x.stride(1), x.stride(2)))
    with torch.cuda.device(img_q.device):
        _hunyuan_qkv_rope_pack_kernel[

View on GitHub (pinned to 0132848349)

Solutions

1. Regenerate cos/sin as 2-D [max_seq, head_dim//2] tensors.
2. Ensure cos and sin come from the same computation so shapes match.
3. Check ndim: squeeze any leading batch dims if the table is [1, S, D/2].

Example fix

// before
cos = freqs_cos.unsqueeze(0)  # [1, S, D/2], ndim=3
// after
cos = freqs_cos.squeeze(0)  # [S, D/2]
sin = freqs_sin.squeeze(0)

assert cos.ndim == 2 and sin.ndim == 2 and cos.shape == sin.shape, (cos.shape, sin.shape)

When it happens

Trigger: Passing cos/sin of different shapes, 1-D precomputed freqs, or 3-D broadcast tables to hunyuan_qkv_rope_pack.

Common situations: Reusing a half-split RoPE table (shape [S, head_dim]) meant for a different kernel, or cos built with head_dim columns instead of head_dim/2.

Related errors

• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D
• rope_pool_fused expects pool tensors to be 3-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/df5594c138746676. Report an issue: GitHub.