sgl-project/sglang · error · ValueError

attention group range [{group_start}, {group_end}) is outsid

Error message

attention group range [{group_start}, {group_end}) is outside consumer_count={self.consumer_count}

What it means

Raised when the rank's derived attention-group slot window [cp_rank * tp_size, ...+tp_size) is not a valid subrange of the pool's consumer_count. This means the parallel state (attn_cp_rank, attn_tp_size) is inconsistent with the number of consumers the pool was created with — e.g. more ranks than pool slots.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:767

    def _pool(self, device_index: int) -> _ImportedCudaVmmPool:
        return _get_imported_pool(
            fabric_handle=self.fabric_handle,
            posix_socket_path=self.posix_socket_path,
            allocation_size=self.allocation_size,
            device_index=device_index,
        )

    def _acknowledgement_range(self, consumer_count: int) -> tuple[int, int]:
        if consumer_count <= 0:
            raise ValueError("consumer_count must be positive")
        if consumer_count == self.consumer_count:
            return 0, self.consumer_count

        parallel = get_parallel()
        group_start = parallel.attn_cp_rank * parallel.attn_tp_size
        group_end = group_start + parallel.attn_tp_size
        if not 0 <= group_start < group_end <= self.consumer_count:
            raise ValueError(
                "attention group range "
                f"[{group_start}, {group_end}) is outside "
                f"consumer_count={self.consumer_count}"
            )
        if consumer_count == 1:
            slot = group_start + parallel.attn_tp_rank
            return slot, slot + 1
        if consumer_count == parallel.attn_tp_size:
            return group_start, group_end
        raise ValueError(
            "consumer_count must be 1, the attention TP size, or the full "
            f"consumer count ({self.consumer_count}); got {consumer_count}"
        )

    def _resolve_consumer_count(self, consumer_count: int | None) -> int:
        return 1 if consumer_count is None else consumer_count

    def _acknowledge_consumption(self, device_index: int, consumer_count: int) -> None:

View on GitHub (pinned to 0132848349)

Solutions

1. Recreate the VMM transport with consumer_count equal to the actual number of participating ranks (cp_size * tp_size)
2. Verify get_parallel() state matches the server_args used to size the pool
3. Ensure all ranks construct the pool with the same consumer_count

from sglang.srt.utils import get_parallel
p = get_parallel()
expected = (p.attn_cp_size if hasattr(p, 'attn_cp_size') else p.attn_cp_rank + 1) * p.attn_tp_size
assert expected <= pool.consumer_count

def group_fits_pool(pool, parallel) -> bool:
    end = parallel.attn_cp_rank * parallel.attn_tp_size + parallel.attn_tp_size
    return 0 < end <= pool.consumer_count

When it happens

Trigger: Consumer pool created with consumer_count smaller than cp_size * tp_size; attn_cp_rank/attn_tp_size read from a parallel state initialized with different world size than the transport pool; mixed single-group and multi-group acknowledgement paths.

Common situations: TP/CP world-size mismatch between exporter and importers; test harness constructing pools with a hardcoded consumer count; reranking of ranks after a restart with stale pool config.

Related errors

• consumer_count must be 1, the attention TP size, or the full
• Decode context parallel (decode_context_parallel_size > 1) i
• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor
• memory_size must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/50444623e6ee7d6b. Report an issue: GitHub.