sgl-project/sglang · error · ValueError

memory_size must be positive

Error message

memory_size must be positive

What it means

Constructor guard on the CUDA VMM shared pool: memory_size must be > 0. Zero/negative sizes are rejected before any driver allocation is attempted.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_transport_utils.py:182

def get_vmm_feature_consumer_count() -> int:
    if get_parallel().enable_dp_attention:
        return get_parallel().tp_size // get_parallel().dp_size
    return get_parallel().tp_size


class CudaVmmMemoryPool:
    """Bounded CUDA VMM pool shared through FABRIC or a local POSIX FD."""

    def __init__(
        self,
        memory_size: int,
        recycle_interval: float,
        base_gpu_id: int,
        consumer_count: int,
        allow_posix_fallback: bool = False,
    ) -> None:
        if memory_size <= 0:
            raise ValueError("memory_size must be positive")
        if consumer_count <= 0:
            raise ValueError("consumer_count must be positive")
        if recycle_interval <= 0:
            raise ValueError("recycle_interval must be positive")

        self.device_index = int(base_gpu_id)
        self.consumer_count = int(consumer_count)
        self._recycle_interval = float(recycle_interval)
        self._lock = threading.Lock()
        self._publisher_condition = threading.Condition(self._lock)
        self._shutdown_lock = threading.Lock()
        self._active_publishers = 0
        self._closing = False
        self._pool_full_warned = False
        self._stop_recycler = threading.Event()
        self._pool_error: BaseException | None = None
        self._closed = False

View on GitHub (pinned to 0132848349)

Solutions

1. Compute and log memory_size before constructing; fix the sizing formula/config
2. Ensure consumer_count and per-consumer bytes are positive integers
3. Set a sane minimum (>= CUDA allocation granularity, see 6377)

Example fix

# before
pool = CudaVmmTransportPool(memory_size=0, ...)
# after
pool = CudaVmmTransportPool(memory_size=512 * 1024 * 1024, ...)

if memory_size <= 0:
    raise ConfigError(f"memory_size={memory_size}")

When it happens

Trigger: Instantiating the VMM pool with memory_size=0 or negative — usually derived from a computed size (e.g. per-consumer bytes * consumer_count) that underflowed or a config value of 0.

Common situations: Misconfigured multimodal transport memory settings, integer underflow in size math, or a default of 0 leaking from config parsing.

Related errors

• consumer_count must be positive
• recycle_interval must be positive
• consumer_count must be 1, the attention TP size, or the full
• CUDA VMM feature transport requires each feature field to co
• Invalid graph capture input size: {nbytes}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e2747c94268596f4. Report an issue: GitHub.