{"record":{"id":"ae01e16d380aa331","repo":"sgl-project/sglang","slug":"positions-slots-must-have-one-entry-per-token","errorCode":null,"errorMessage":"positions/slots must have one entry per token","messagePattern":"positions/slots must have one entry per token","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/metal.py","lineNumber":88,"sourceCode":"    k_shape = tuple(k.shape)\n    v_shape = tuple(v.shape)\n    positions_shape = tuple(positions.shape)\n    slots_shape = tuple(slots.shape)\n    k_pool_shape = tuple(k_pool.shape)\n    v_pool_shape = tuple(v_pool.shape)\n\n    if q_shape != (q_shape[0], num_qo_heads, head_dim):\n        raise ValueError(\n            \"q shape must be [num_tokens, num_qo_heads, head_dim], \" f\"got {q.shape}\"\n        )\n    if k_shape != (q_shape[0], num_kv_heads, head_dim):\n        raise ValueError(\n            \"k shape must be [num_tokens, num_kv_heads, head_dim], \" f\"got {k.shape}\"\n        )\n    if v_shape != k_shape:\n        raise ValueError(f\"v shape must match k shape, got {v.shape} vs {k.shape}\")\n    if positions_shape != (q_shape[0],) or slots_shape != (q_shape[0],):\n        raise ValueError(\"positions/slots must have one entry per token\")\n    if k_pool_shape[1:] != (num_kv_heads, head_dim):\n        raise ValueError(f\"k_pool has incompatible shape {k_pool.shape}\")\n    if v_pool_shape != k_pool_shape:\n        raise ValueError(\n            f\"v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}\"\n        )\n    if q.dtype != k.dtype or q.dtype != v.dtype:\n        raise ValueError(\"q/k/v dtypes must match\")\n    if k_pool.dtype != q.dtype or v_pool.dtype != q.dtype:\n        raise ValueError(\"pool dtypes must match q/k/v dtype\")\n\n    return _metal.rope_pool_fused(\n        q,\n        k,\n        v,\n        positions,\n        slots,\n        k_pool,","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/metal.py#L70-L106","documentation":"The per-token positions and pool slot index tensors must each have exactly num_tokens entries (length equal to q.shape[0]). This error fires when their length disagrees with the token count of q/k/v, meaning the kernel cannot map each token to its rope position and cache slot.","triggerScenarios":"Calling rope_pool_fused where len(positions) != q.shape[0] or len(slots) != q.shape[0], e.g. positions computed for the full sequence while q holds only a chunk, or slots allocated for a different batch size.","commonSituations":"Chunked prefill that slices q/k/v but forgets to slice positions/slots; scheduler passing cumulated positions for previous tokens; cache slot tensor computed for a different concurrency level.","solutions":["Slice positions/slots to the token count: positions = positions[: q.shape[0]]; slots = slots[: q.shape[0]]","Compute positions per chunk from the last context length: torch.arange(last_len, last_len + chunk_len)","Assert positions.shape[0] == slots.shape[0] == q.shape[0] before the call"],"exampleFix":"# before\nq, k, v = q[chunk:], k[chunk:], v[chunk:]\nmetal.rope_pool_fused(q, k, v, positions, slots, ...)  # full-length positions\n\n# after\nq, k, v = q[chunk:], k[chunk:], v[chunk:]\npositions = positions[chunk:]\nslots = slots[chunk:]\nmetal.rope_pool_fused(q, k, v, positions, slots, ...)","handlingStrategy":"validation","validationCode":"assert positions.shape == slots.shape == (q.shape[0],)","typeGuard":"def per_token_indices(q, positions, slots) -> bool:\n    return positions.shape == slots.shape == (q.shape[0],)","tryCatchPattern":null,"preventionTips":["Slice positions/slocks whenever you slice q/k/v","Compute chunk positions from last_len + arange(chunk_len)"],"tags":["shape-validation","positions","kv-cache-slots","rope","sgl-kernel"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}