sgl-project/sglang · error · ValueError

head_dim must be positive, even, and <= 128

Error message

head_dim must be positive, even, and <= 128

What it means

The Hunyuan QKV RoPE pack kernel only supports head_dim values that are positive, even, and at most 128. The even requirement comes from RoPE splitting the head into two halves, and 128 is the Triton kernel's BLOCK size limit.

Source

Thrown at python/sglang/kernels/ops/diffusion/rope/hunyuan_qkv_pack_triton.py:179

    tensors = (img_q, img_k, img_v, txt_q, txt_k, txt_v)
    if any(x.ndim != 4 for x in tensors):
        raise ValueError("QKV tensors must have shape [B, S, H, D]")
    if any(not x.is_cuda or x.dtype != torch.bfloat16 for x in tensors):
        raise ValueError("QKV tensors must be CUDA bfloat16 tensors")
    if any(x.device != img_q.device for x in tensors):
        raise ValueError("QKV tensors must be on the same CUDA device")
    batch, img_tokens, num_heads, head_dim = img_q.shape
    txt_tokens = txt_q.shape[1]
    expected_img = (batch, img_tokens, num_heads, head_dim)
    expected_txt = (batch, txt_tokens, num_heads, head_dim)
    if any(tuple(x.shape) != expected_img for x in (img_q, img_k, img_v)):
        raise ValueError("image QKV shapes must match")
    if any(tuple(x.shape) != expected_txt for x in (txt_q, txt_k, txt_v)):
        raise ValueError("text QKV shapes must match")
    if any(x.stride(-1) != 1 for x in tensors):
        raise ValueError("QKV last dimensions must be contiguous")
    if head_dim <= 0 or head_dim > 128 or head_dim % 2:
        raise ValueError("head_dim must be positive, even, and <= 128")
    if cos.ndim != 2 or sin.ndim != 2 or cos.shape != sin.shape:
        raise ValueError("cos and sin must have matching [S, D/2] shapes")
    if cos.shape[0] < img_tokens or cos.shape[1] != head_dim // 2:
        raise ValueError("cos/sin shape does not cover image tokens and head_dim")
    if not cos.is_cuda or not sin.is_cuda or cos.stride(-1) != 1 or sin.stride(-1) != 1:
        raise ValueError("cos and sin must be CUDA and last-dim contiguous")
    if cos.device != img_q.device or sin.device != img_q.device:
        raise ValueError("QKV and cos/sin tensors must be on the same CUDA device")

    total_tokens = img_tokens + txt_tokens
    storage = torch.empty(
        (3, batch, total_tokens, num_heads, head_dim),
        device=img_q.device,
        dtype=img_q.dtype,
    )
    args = []
    for x in tensors:
        args.extend((x.stride(0), x.stride(1), x.stride(2)))

View on GitHub (pinned to 0132848349)

Solutions

1. Verify the model config's head_dim (hidden_size / num_heads) is even and <= 128.
2. If head_dim > 128, split the head and call the kernel per chunk or use a fallback path.
3. Fix the projection layer that produced the wrong head_dim.

Example fix

// before
img_q = proj(x)  # shape [..., 100] -> head_dim=100 (odd)
hunyuan_qkv_rope_pack(...)
// after
assert img_q.shape[-1] % 2 == 0 and img_q.shape[-1] <= 128, img_q.shape

head_dim = img_q.shape[-1]
assert head_dim > 0 and head_dim <= 128 and head_dim % 2 == 0, head_dim

When it happens

Trigger: Calling hunyuan_qkv_rope_pack with head_dim inferred from an odd or >128 innermost dimension, e.g. a malformed projection layer outputting head_dim=100 or head_dim=256.

Common situations: Misconfigured attention head counts (num_heads*head_dim mismatch with the QKV projection width), or porting a model whose head_dim exceeds the kernel's 128 cap.

Related errors

• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D
• rope_pool_fused expects pool tensors to be 3-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/40409c05aaa0df53. Report an issue: GitHub.