sgl-project/sglang · error · ValueError

consumer_count must be 1, the attention TP size, or the full

Error message

consumer_count must be 1, the attention TP size, or the full consumer count ({self.consumer_count}); got {consumer_count}

What it means

The acknowledgement protocol only supports three granularities: acknowledging a single consumer slot (1), the rank's whole attention TP group (attn_tp_size), or every consumer (full consumer_count). Any other consumer_count is rejected because there is no defined slot range for it.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:777

            raise ValueError("consumer_count must be positive")
        if consumer_count == self.consumer_count:
            return 0, self.consumer_count

        parallel = get_parallel()
        group_start = parallel.attn_cp_rank * parallel.attn_tp_size
        group_end = group_start + parallel.attn_tp_size
        if not 0 <= group_start < group_end <= self.consumer_count:
            raise ValueError(
                "attention group range "
                f"[{group_start}, {group_end}) is outside "
                f"consumer_count={self.consumer_count}"
            )
        if consumer_count == 1:
            slot = group_start + parallel.attn_tp_rank
            return slot, slot + 1
        if consumer_count == parallel.attn_tp_size:
            return group_start, group_end
        raise ValueError(
            "consumer_count must be 1, the attention TP size, or the full "
            f"consumer count ({self.consumer_count}); got {consumer_count}"
        )

    def _resolve_consumer_count(self, consumer_count: int | None) -> int:
        return 1 if consumer_count is None else consumer_count

    def _acknowledge_consumption(self, device_index: int, consumer_count: int) -> None:
        if self._consumer_acknowledged:
            return
        pool = self._pool(device_index)
        ack_start = self.control_offset
        ack_end = ack_start + self.consumer_count * _CONTROL_WORD_BYTES
        ack_words = pool.memory[ack_start:ack_end].view(torch.int32)
        slot_start, slot_end = self._acknowledgement_range(consumer_count)
        # This kernel is ordered after the remote read on the consumer stream;
        # observing the flag therefore means the pool slice is safe to reuse.
        ack_words[slot_start:slot_end].fill_(1)

View on GitHub (pinned to 0132848349)

Solutions

1. Use consumer_count=None (single slot) or pass parallel.attn_tp_size / the pool's full consumer_count
2. Derive the count from get_parallel() rather than hardcoding
3. After changing TP size, rebuild the transport so consumer_count matches

p = get_parallel()
valid = {1, p.attn_tp_size, pool.consumer_count}
if consumer_count not in valid:
    consumer_count = None  # or full count

def is_supported_ack_count(n: int, pool, parallel) -> bool:
    return n in (1, parallel.attn_tp_size, pool.consumer_count)

When it happens

Trigger: Calling acknowledge_consumption(consumer_count=N) where N is not 1, not attn_tp_size, and not the pool's consumer_count; e.g. acknowledging a partial subgroup or an outdated tp size.

Common situations: Code written for a different tp configuration reused after resizing; callers guessing a count instead of deriving it from parallel state.

Related errors

• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive
• attention group range [{group_start}, {group_end}) is outsid
• CUDA VMM feature transport requires each feature field to co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/366eee760c71eb7f. Report an issue: GitHub.