{"record":{"id":"4afbee81f76e1e80","repo":"sgl-project/sglang","slug":"qwen-vl-position-ids-do-not-match-the-attention-in","errorCode":null,"errorMessage":"Qwen-VL position_ids do not match the attention input","messagePattern":"Qwen-VL position_ids do not match the attention input","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/encoders/qwen_vl_rope.py","lineNumber":57,"sourceCode":"    if query.ndim != 4 or key.ndim != 4:\n        raise ValueError(\n            \"Qwen-VL query and key must have shape \"\n            \"[batch, heads, sequence, head_dim]\"\n        )\n    if position_ids.ndim != 3 or position_ids.shape[0] != 3:\n        raise ValueError(\n            \"Qwen-VL text position_ids must have shape [3, batch, sequence]\"\n        )\n    batch_size, num_query_heads, sequence_length, head_dim = query.shape\n    key_batch_size, num_key_value_heads, key_sequence_length, key_head_dim = key.shape\n    if (key_batch_size, key_sequence_length, key_head_dim) != (\n        batch_size,\n        sequence_length,\n        head_dim,\n    ):\n        raise ValueError(\"Qwen-VL query and key shapes are incompatible\")\n    if tuple(position_ids.shape[1:]) != (batch_size, sequence_length):\n        raise ValueError(\"Qwen-VL position_ids do not match the attention input\")\n\n    query = query.transpose(1, 2).reshape(-1, num_query_heads * head_dim)\n    key = key.transpose(1, 2).reshape(-1, num_key_value_heads * head_dim)\n    # Preserve HF's bf16 arithmetic order; fused MRoPE changes generated images.\n    query, key = rotary_emb.forward_native(position_ids.reshape(3, -1), query, key)\n    query = query.view(batch_size, sequence_length, num_query_heads, head_dim)\n    key = key.view(batch_size, sequence_length, num_key_value_heads, head_dim)\n    return query.transpose(1, 2), key.transpose(1, 2)\n","sourceCodeStart":39,"sourceCodeEnd":66,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/encoders/qwen_vl_rope.py#L39-L66","documentation":"Thrown by apply_qwen_vl_text_rope when the shape of position_ids does not match the (batch_size, sequence_length) dimensions derived from the query tensor. Qwen-VL multimodal RoPE requires a 3D position_ids tensor whose trailing dims align exactly with the attention input layout before the rotary embedding is applied.","triggerScenarios":"Calling apply_qwen_vl_text_rope (or the encoder's forward) with position_ids of shape (3, B, S) whose dims [1:] differ from the batch_size/sequence_length inferred from query, e.g. padding added to query/key but not position_ids, or transposed (B,3,S) layout instead of (3,B,S).","commonSituations":"Batched GQA layouts where heads reshape query but seq_len/batch get mixed up; passing HF-style position_ids without re-batching; running the two referenced tests with mismatched dummy shapes; changes in padding/bucketing that update one tensor but not the other.","solutions":["Verify position_ids.shape == (3, batch_size, sequence_length) matches query.shape[0] and query.shape[1] (or the transposed layout the function expects) before calling","Rebuild position_ids from the same batch/seq metadata used to build query/key instead of caching them separately","If you transposed query to (B, S, H, D) yourself, ensure position_ids was permuted consistently","Check for off-by-one padding: attention may include extra tokens (e.g. vision tokens) not reflected in position_ids"],"exampleFix":"// before\npos_ids = torch.arange(seq_len).expand(3, 1, seq_len_padded)  # wrong seq len\nquery, key = apply_qwen_vl_text_rope(query, key, pos_ids, rotary_emb)\n// after\nassert pos_ids.shape[1:] == (query.shape[0], query.shape[1]), (pos_ids.shape, query.shape)\nquery, key = apply_qwen_vl_text_rope(query, key, pos_ids, rotary_emb)","handlingStrategy":"validation","validationCode":"bs, sl = query.shape[0], query.shape[1]\nassert position_ids.ndim == 3 and position_ids.shape[1:] == (bs, sl), \\\n    f\"position_ids {tuple(position_ids.shape)} != {(3, bs, sl)}\"","typeGuard":"def valid_qwen_vl_pos_ids(position_ids: torch.Tensor, bs: int, sl: int) -> bool:\n    return position_ids.ndim == 3 and position_ids.shape[1:] == (bs, sl)","tryCatchPattern":"try:\n    q, k = apply_qwen_vl_text_rope(q, k, pos, rotary)\nexcept ValueError as e:\n    raise ValueError(f\"RoPE layout mismatch: pos={tuple(pos.shape)} q={tuple(q.shape)}\") from e","preventionTips":["Derive position_ids from the same batch/seq metadata used to build query/key","Add a shape assert in test fixtures (the two referenced tests encode the expected layout)","Keep the (3, B, S) axis order explicit when constructing M-RoPE ids"],"tags":["qwen-vl","rope","position-ids","shape-mismatch","multimodal"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}