sgl-project/sglang · error · ValueError

positions/slots must have one entry per token

Error message

positions/slots must have one entry per token

What it means

The per-token positions and pool slot index tensors must each have exactly num_tokens entries (length equal to q.shape[0]). This error fires when their length disagrees with the token count of q/k/v, meaning the kernel cannot map each token to its rope position and cache slot.

Source

Thrown at python/sglang/kernels/aot/python/sgl_kernel/metal.py:88

    k_shape = tuple(k.shape)
    v_shape = tuple(v.shape)
    positions_shape = tuple(positions.shape)
    slots_shape = tuple(slots.shape)
    k_pool_shape = tuple(k_pool.shape)
    v_pool_shape = tuple(v_pool.shape)

    if q_shape != (q_shape[0], num_qo_heads, head_dim):
        raise ValueError(
            "q shape must be [num_tokens, num_qo_heads, head_dim], " f"got {q.shape}"
        )
    if k_shape != (q_shape[0], num_kv_heads, head_dim):
        raise ValueError(
            "k shape must be [num_tokens, num_kv_heads, head_dim], " f"got {k.shape}"
        )
    if v_shape != k_shape:
        raise ValueError(f"v shape must match k shape, got {v.shape} vs {k.shape}")
    if positions_shape != (q_shape[0],) or slots_shape != (q_shape[0],):
        raise ValueError("positions/slots must have one entry per token")
    if k_pool_shape[1:] != (num_kv_heads, head_dim):
        raise ValueError(f"k_pool has incompatible shape {k_pool.shape}")
    if v_pool_shape != k_pool_shape:
        raise ValueError(
            f"v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}"
        )
    if q.dtype != k.dtype or q.dtype != v.dtype:
        raise ValueError("q/k/v dtypes must match")
    if k_pool.dtype != q.dtype or v_pool.dtype != q.dtype:
        raise ValueError("pool dtypes must match q/k/v dtype")

    return _metal.rope_pool_fused(
        q,
        k,
        v,
        positions,
        slots,
        k_pool,

View on GitHub (pinned to 0132848349)

Solutions

1. Slice positions/slots to the token count: positions = positions[: q.shape[0]]; slots = slots[: q.shape[0]]
2. Compute positions per chunk from the last context length: torch.arange(last_len, last_len + chunk_len)
3. Assert positions.shape[0] == slots.shape[0] == q.shape[0] before the call

Example fix

# before
q, k, v = q[chunk:], k[chunk:], v[chunk:]
metal.rope_pool_fused(q, k, v, positions, slots, ...)  # full-length positions

# after
q, k, v = q[chunk:], k[chunk:], v[chunk:]
positions = positions[chunk:]
slots = slots[chunk:]
metal.rope_pool_fused(q, k, v, positions, slots, ...)

assert positions.shape == slots.shape == (q.shape[0],)

def per_token_indices(q, positions, slots) -> bool:
    return positions.shape == slots.shape == (q.shape[0],)

When it happens

Trigger: Calling rope_pool_fused where len(positions) != q.shape[0] or len(slots) != q.shape[0], e.g. positions computed for the full sequence while q holds only a chunk, or slots allocated for a different batch size.

Common situations: Chunked prefill that slices q/k/v but forgets to slice positions/slots; scheduler passing cumulated positions for previous tokens; cache slot tensor computed for a different concurrency level.

Related errors

• rope_pool_fused expects positions/slots to be 1-D
• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects pool tensors to be 3-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ae01e16d380aa331. Report an issue: GitHub.