sgl-project/sglang · error · ValueError

total_pool_size must be positive

Error message

total_pool_size must be positive

What it means

get_mm_feature_pool_size_per_worker divides a configured CUDA IPC multimodal feature pool budget across tokenizer workers and requires the total byte budget to be strictly positive. Zero or negative sizes (misconfigured --mm-...-memory-budget, or an off-by-suffix like bytes vs MB producing 0 after int truncation) fail fast before any GPU allocation.

Source

Thrown at python/sglang/srt/multimodal/transport/cuda_ipc.py:42

# model has computed the data-parallel assignment.
DEFER_CUDA_IPC_FEATURE_RECONSTRUCTION_KEY = (
    "_sglang_defer_cuda_ipc_feature_reconstruction"
)


def get_mm_feature_pool_size_per_worker(
    total_pool_size: int, tokenizer_worker_num: int
) -> int:
    """Split the CUDA IPC feature-pool budget without exceeding it.

    Each tokenizer worker owns a distinct CUDA allocation, even though all pools
    are created on ``base_gpu_id``.  Therefore a minimum per-worker allocation
    would make the aggregate HBM reservation larger than the configured budget.
    Keep the configured value as a hard per-node cap and leave at most
    ``tokenizer_worker_num - 1`` bytes unused when it is not evenly divisible.
    """
    if total_pool_size <= 0:
        raise ValueError("total_pool_size must be positive")
    if tokenizer_worker_num <= 0:
        raise ValueError("tokenizer_worker_num must be positive")

    return total_pool_size // tokenizer_worker_num


# Cache for pool-level IPC handles on the consumer side.
# Key: the pool CUDA IPC handle tuple. Value: opened UntypedStorage.
_pool_storage_cache: dict = {}
_pool_cache_lock = threading.Lock()


def _normalize_pool_cache_key(pool_handle, device_index: int) -> tuple[Any, ...]:
    normalized_handle = (
        pool_handle if isinstance(pool_handle, tuple) else tuple(pool_handle)
    )
    return (device_index, normalized_handle)

View on GitHub (pinned to 0132848349)

Solutions

1. Set a positive budget (e.g. 1GB in bytes) in the server args for the mm feature pool
2. Check the unit parsing of the budget flag — ensure MB/GB conversion doesn't floor to 0
3. If you intended to disable the pool, use the supported disable mechanism rather than 0

Example fix

# before
--cuda-ipc-mm-feature-pool-size 0
# after
--cuda-ipc-mm-feature-pool-size 1073741824

if total_pool_size is None or int(total_pool_size) <= 0:
    raise ValueError('mm feature pool size must be a positive byte count')

When it happens

Trigger: Calling the CUDA IPC transport init (or its unit tests) with total_pool_size <= 0, e.g. budget string '0' parsed to 0 bytes, or negative value from config arithmetic.

Common situations: Setting the mm feature pool memory budget to 0 to 'disable' it instead of using the proper disable flag; parsing sizes in the wrong unit so integer division floors to 0; env var typos defaulting to 0.

Related errors

• tokenizer_worker_num must be positive
• memory_size must be positive
• consumer_count must be positive
• max_inflight_slices must be positive
• total_consumer_count must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dbd843026a0b21bc. Report an issue: GitHub.