sgl-project/sglang · error · ValueError

consumer_count must be positive

Error message

consumer_count must be positive

What it means

The transport memory pool requires consumer_count > 0 because it allocates ready/ack slots per consumer. A non-positive count would leave the pool with no valid acknowledgement topology, so the constructor raises ValueError.

Source

Thrown at python/sglang/srt/multimodal/transport/memory_pool.py:174

class StreamOrderedMmFeaturePool:
    """Bounded GPU pool with generation-safe producer/consumer leases."""

    def __init__(
        self,
        *,
        memory_size: int,
        byte_tensor: torch.Tensor,
        base_address: int,
        device_id: int,
        consumer_count: int,
        recycle_interval: float,
        transport_name: str,
        max_inflight_slices: int = DEFAULT_MAX_INFLIGHT_SLICES,
    ) -> None:
        if memory_size <= 0:
            raise ValueError("memory_size must be positive")
        if consumer_count <= 0:
            raise ValueError("consumer_count must be positive")
        if max_inflight_slices <= 0:
            raise ValueError("max_inflight_slices must be positive")
        if recycle_interval <= 0:
            raise ValueError("recycle_interval must be positive")
        if (
            not byte_tensor.is_cuda
            or byte_tensor.device.index != device_id
            or byte_tensor.dtype != torch.uint8
            or not byte_tensor.is_contiguous()
            or byte_tensor.numel() < memory_size
        ):
            raise ValueError(
                "byte_tensor must be a sufficiently large contiguous uint8 tensor "
                f"on cuda:{device_id}"
            )

        self.memory_size = memory_size
        self.byte_tensor = byte_tensor

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure at least one consumer (tokenizer worker) is configured
2. Fix --tokenizer-worker-num / DP configuration
3. Construct the transport only after worker topology is known

Example fix

# before
MemoryPoolTransport(..., consumer_count=0, ...)
# after
MemoryPoolTransport(..., consumer_count=tokenizer_worker_num, ...)  # >=1

assert isinstance(consumer_count, int) and consumer_count >= 1

When it happens

Trigger: Constructing the pool with consumer_count=0 — typically a derived value from tokenizer worker count or DP size that computed to zero.

Common situations: Server args yielding zero tokenizer consumers; startup before topology resolution; test harnesses passing placeholder 0.

Related errors

• total_pool_size must be positive
• tokenizer_worker_num must be positive
• memory_size must be positive
• max_inflight_slices must be positive
• total_consumer_count must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3460fdd09ff6ea30. Report an issue: GitHub.