sgl-project/sglang · error · ValueError

total_consumer_count must be positive

Error message

total_consumer_count must be positive

What it means

StreamOrderedConsumer initialization requires total_consumer_count > 0 because ready/ack bookkeeping allocates one slot per consumer. Zero or negative counts would make acknowledgement arithmetic and slot offsets invalid, so it fails fast with ValueError.

Solutions

1. Ensure the tokenizer worker / consumer count is at least 1 before starting the server
2. Fix server args (--tokenizer-worker-num, --dp-size) so computed consumer count > 0
3. Delay transport construction until worker topology is resolved

Example fix

# before
_init_stream_ordered_consumer(..., total_consumer_count=0, ...)
# after
_init_stream_ordered_consumer(..., total_consumer_count=num_tokenizer_workers, ...)  # >=1

if total_consumer_count is None or total_consumer_count < 1:
    raise ValueError('consumer count must be >= 1')

When it happens

Trigger: Constructing the IPC transport (whose __init__ calls _init_stream_ordered_consumer) with total_consumer_count=0 — typically because the tokenizer worker count resolved to 0 at startup.

Common situations: Degenerate configs (dp=1 with 0 tokenizer workers computed), early-startup races where worker counts are not yet populated, or tests passing 0 explicitly.

Related errors

• consumer_count must be positive
• max_inflight_slices must be positive
• memory_size must be positive
• tokenizer_worker_num must be positive
• total_pool_size must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7a15b1eb0305e969. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/multimodal/transport/memory_pool.py:93

            f"[0, {total_consumer_count})"
        )
    return rank


class StreamOrderedPoolConsumerMixin:
    """Ready/wait/ack protocol for stream-ordered GPU feature proxies."""

    def _init_stream_ordered_consumer(
        self,
        *,
        ready_byte_offset: int,
        ack_byte_offset: int,
        generation: int,
        total_consumer_count: int,
        transport_name: str,
    ) -> None:
        if total_consumer_count <= 0:
            raise ValueError("total_consumer_count must be positive")
        self.ready_byte_offset = ready_byte_offset
        self.ack_byte_offset = ack_byte_offset
        self.generation = generation
        self.total_consumer_count = total_consumer_count
        self.transport_name = transport_name
        self._consumer_acknowledged = False

    def _wait_until_ready(self, base_address: int, device_id: int) -> None:
        stream_wait_value32(
            device_id,
            base_address + self.ready_byte_offset,
            self.generation,
            self.transport_name,
        )

    def _acknowledge_on_stream(
        self,
        base_address: int,

View on GitHub (pinned to 0132848349)