sgl-project/sglang · error · ValueError

max_inflight_slices must be positive

Error message

max_inflight_slices must be positive

What it means

The pool caps concurrent in-flight slices (DEFAULT_MAX_INFLIGHT_SLICES) to bound ready/ack generation bookkeeping; max_inflight_slices must be >= 1. Zero or negative values would deadlock recycling since no slice could ever be in flight.

Source

Thrown at python/sglang/srt/multimodal/transport/memory_pool.py:176

    def __init__(
        self,
        *,
        memory_size: int,
        byte_tensor: torch.Tensor,
        base_address: int,
        device_id: int,
        consumer_count: int,
        recycle_interval: float,
        transport_name: str,
        max_inflight_slices: int = DEFAULT_MAX_INFLIGHT_SLICES,
    ) -> None:
        if memory_size <= 0:
            raise ValueError("memory_size must be positive")
        if consumer_count <= 0:
            raise ValueError("consumer_count must be positive")
        if max_inflight_slices <= 0:
            raise ValueError("max_inflight_slices must be positive")
        if recycle_interval <= 0:
            raise ValueError("recycle_interval must be positive")
        if (
            not byte_tensor.is_cuda
            or byte_tensor.device.index != device_id
            or byte_tensor.dtype != torch.uint8
            or not byte_tensor.is_contiguous()
            or byte_tensor.numel() < memory_size
        ):
            raise ValueError(
                "byte_tensor must be a sufficiently large contiguous uint8 tensor "
                f"on cuda:{device_id}"
            )

        self.memory_size = memory_size
        self.byte_tensor = byte_tensor
        self.base_address = base_address
        self.device_id = device_id

View on GitHub (pinned to 0132848349)

Solutions

1. Set max_inflight_slices to at least 1
2. If limiting memory, lower slice size instead of slice count
3. Validate computed values before passing them in

Example fix

# before
MemoryPoolTransport(..., max_inflight_slices=0, ...)
# after
MemoryPoolTransport(..., max_inflight_slices=1, ...)

if max_inflight_slices is not None:
    assert int(max_inflight_slices) >= 1, 'max_inflight_slices must be >= 1'

When it happens

Trigger: Constructing the transport pool with max_inflight_slices <= 0 — explicit override with 0 or a computed concurrency value that rounded down to 0.

Common situations: Tuning flags for low memory by setting inflight slices to 0 instead of 1; deriving the value from batch size / concurrency math that can produce 0; copy-pasted configs.

Related errors

• total_pool_size must be positive
• tokenizer_worker_num must be positive
• memory_size must be positive
• consumer_count must be positive
• total_consumer_count must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cf885d1d9e3327ea. Report an issue: GitHub.