sgl-project/sglang · error · ValueError

QKV last dimensions must be contiguous

Error message

QKV last dimensions must be contiguous

What it means

The Hunyuan QKV RoPE pack Triton kernel requires every Q/K/V tensor (image and text) to have unit stride in the last dimension (contiguous head_dim). The Triton kernel indexes head elements assuming a dense innermost dimension, so non-contiguous inputs would read wrong memory.

Source

Thrown at python/sglang/kernels/ops/diffusion/rope/hunyuan_qkv_pack_triton.py:177

    sin: torch.Tensor,
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
    tensors = (img_q, img_k, img_v, txt_q, txt_k, txt_v)
    if any(x.ndim != 4 for x in tensors):
        raise ValueError("QKV tensors must have shape [B, S, H, D]")
    if any(not x.is_cuda or x.dtype != torch.bfloat16 for x in tensors):
        raise ValueError("QKV tensors must be CUDA bfloat16 tensors")
    if any(x.device != img_q.device for x in tensors):
        raise ValueError("QKV tensors must be on the same CUDA device")
    batch, img_tokens, num_heads, head_dim = img_q.shape
    txt_tokens = txt_q.shape[1]
    expected_img = (batch, img_tokens, num_heads, head_dim)
    expected_txt = (batch, txt_tokens, num_heads, head_dim)
    if any(tuple(x.shape) != expected_img for x in (img_q, img_k, img_v)):
        raise ValueError("image QKV shapes must match")
    if any(tuple(x.shape) != expected_txt for x in (txt_q, txt_k, txt_v)):
        raise ValueError("text QKV shapes must match")
    if any(x.stride(-1) != 1 for x in tensors):
        raise ValueError("QKV last dimensions must be contiguous")
    if head_dim <= 0 or head_dim > 128 or head_dim % 2:
        raise ValueError("head_dim must be positive, even, and <= 128")
    if cos.ndim != 2 or sin.ndim != 2 or cos.shape != sin.shape:
        raise ValueError("cos and sin must have matching [S, D/2] shapes")
    if cos.shape[0] < img_tokens or cos.shape[1] != head_dim // 2:
        raise ValueError("cos/sin shape does not cover image tokens and head_dim")
    if not cos.is_cuda or not sin.is_cuda or cos.stride(-1) != 1 or sin.stride(-1) != 1:
        raise ValueError("cos and sin must be CUDA and last-dim contiguous")
    if cos.device != img_q.device or sin.device != img_q.device:
        raise ValueError("QKV and cos/sin tensors must be on the same CUDA device")

    total_tokens = img_tokens + txt_tokens
    storage = torch.empty(
        (3, batch, total_tokens, num_heads, head_dim),
        device=img_q.device,
        dtype=img_q.dtype,
    )
    args = []

View on GitHub (pinned to 0132848349)

Solutions

1. Make each tensor contiguous before the call: q = q.contiguous() etc.
2. Restructure upstream slicing to produce last-dim-contiguous views (slice leading dims, not the innermost one).
3. Check x.stride(-1) == 1 in a debug assert before calling to find the offending tensor.

Example fix

// before
q, k, v = x[..., ::2], x[..., 1::2]  # stride(-1) == 2
out = hunyuan_qkv_rope_pack(q, k, v, qt, kt, vt, cos, sin)
// after
q, k, v = x[..., ::2].contiguous(), x[..., 1::2].contiguous()
out = hunyuan_qkv_rope_pack(q, k, v, qt, kt, vt, cos, sin)

def qkv_ok(*ts):
    return all(t.stride(-1) == 1 for t in ts)
assert qkv_ok(img_q, img_k, img_v, txt_q, txt_k, txt_v)

try:
    out = hunyuan_qkv_rope_pack(...)
except ValueError as e:
    if 'contiguous' in str(e):
        tensors = [t.contiguous() for t in tensors]
        out = hunyuan_qkv_rope_pack(...)
    else:
        raise

When it happens

Trigger: Calling hunyuan_qkv_rope_pack (directly or via _hunyuan_pack_qkv) with any of img_q/img_k/img_v/txt_q/txt_k/txt_v produced by a non-contiguous operation, e.g. a transpose, narrow, or slice such as x[:, :, ::2] giving stride(-1) != 1.

Common situations: Passing split half tensors from interleaved RoPE-style slicing (x[..., :d/2] is fine but x[..., ::2] is not), or transposed attention projections from a custom attention module.

Related errors

• QKV tensors must have shape [B, S, H, D]
• QKV tensors must be CUDA bfloat16 tensors
• QKV tensors must be on the same CUDA device
• image QKV shapes must match
• text QKV shapes must match

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/18ff79f14027495c. Report an issue: GitHub.