sgl-project/sglang · error · ValueError

Qwen-VL position_ids do not match the attention input

Error message

Qwen-VL position_ids do not match the attention input

What it means

Thrown by apply_qwen_vl_text_rope when the shape of position_ids does not match the (batch_size, sequence_length) dimensions derived from the query tensor. Qwen-VL multimodal RoPE requires a 3D position_ids tensor whose trailing dims align exactly with the attention input layout before the rotary embedding is applied.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/encoders/qwen_vl_rope.py:57

    if query.ndim != 4 or key.ndim != 4:
        raise ValueError(
            "Qwen-VL query and key must have shape "
            "[batch, heads, sequence, head_dim]"
        )
    if position_ids.ndim != 3 or position_ids.shape[0] != 3:
        raise ValueError(
            "Qwen-VL text position_ids must have shape [3, batch, sequence]"
        )
    batch_size, num_query_heads, sequence_length, head_dim = query.shape
    key_batch_size, num_key_value_heads, key_sequence_length, key_head_dim = key.shape
    if (key_batch_size, key_sequence_length, key_head_dim) != (
        batch_size,
        sequence_length,
        head_dim,
    ):
        raise ValueError("Qwen-VL query and key shapes are incompatible")
    if tuple(position_ids.shape[1:]) != (batch_size, sequence_length):
        raise ValueError("Qwen-VL position_ids do not match the attention input")

    query = query.transpose(1, 2).reshape(-1, num_query_heads * head_dim)
    key = key.transpose(1, 2).reshape(-1, num_key_value_heads * head_dim)
    # Preserve HF's bf16 arithmetic order; fused MRoPE changes generated images.
    query, key = rotary_emb.forward_native(position_ids.reshape(3, -1), query, key)
    query = query.view(batch_size, sequence_length, num_query_heads, head_dim)
    key = key.view(batch_size, sequence_length, num_key_value_heads, head_dim)
    return query.transpose(1, 2), key.transpose(1, 2)

View on GitHub (pinned to 0132848349)

Solutions

1. Verify position_ids.shape == (3, batch_size, sequence_length) matches query.shape[0] and query.shape[1] (or the transposed layout the function expects) before calling
2. Rebuild position_ids from the same batch/seq metadata used to build query/key instead of caching them separately
3. If you transposed query to (B, S, H, D) yourself, ensure position_ids was permuted consistently
4. Check for off-by-one padding: attention may include extra tokens (e.g. vision tokens) not reflected in position_ids

Example fix

// before
pos_ids = torch.arange(seq_len).expand(3, 1, seq_len_padded)  # wrong seq len
query, key = apply_qwen_vl_text_rope(query, key, pos_ids, rotary_emb)
// after
assert pos_ids.shape[1:] == (query.shape[0], query.shape[1]), (pos_ids.shape, query.shape)
query, key = apply_qwen_vl_text_rope(query, key, pos_ids, rotary_emb)

bs, sl = query.shape[0], query.shape[1]
assert position_ids.ndim == 3 and position_ids.shape[1:] == (bs, sl), \
    f"position_ids {tuple(position_ids.shape)} != {(3, bs, sl)}"

def valid_qwen_vl_pos_ids(position_ids: torch.Tensor, bs: int, sl: int) -> bool:
    return position_ids.ndim == 3 and position_ids.shape[1:] == (bs, sl)

try:
    q, k = apply_qwen_vl_text_rope(q, k, pos, rotary)
except ValueError as e:
    raise ValueError(f"RoPE layout mismatch: pos={tuple(pos.shape)} q={tuple(q.shape)}") from e

When it happens

Trigger: Calling apply_qwen_vl_text_rope (or the encoder's forward) with position_ids of shape (3, B, S) whose dims [1:] differ from the batch_size/sequence_length inferred from query, e.g. padding added to query/key but not position_ids, or transposed (B,3,S) layout instead of (3,B,S).

Common situations: Batched GQA layouts where heads reshape query but seq_len/batch get mixed up; passing HF-style position_ids without re-batching; running the two referenced tests with mismatched dummy shapes; changes in padding/bucketing that update one tensor but not the other.

Related errors

• img_position_ids must be [1, S, 3], got {list(img_position_i
• cos/sin shape does not cover image tokens and head_dim
• QwenImage RoPE text cache overflow before denoising: require
• vis_freqs_cis must be a 2D cos_sin_cache tensor
• Fused QK-Norm + RoPE kernel only supports float16/bfloat16,

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4afbee81f76e1e80. Report an issue: GitHub.