sgl-project/sglang · error · ValueError

tokenizer_worker_num must be positive

Error message

tokenizer_worker_num must be positive

What it means

The same pool-budget divider requires tokenizer_worker_num to be a positive integer: the per-worker budget is total_pool_size // tokenizer_worker_num, and zero/negative workers makes the division meaningless. Raised before any IPC handle is created.

Source

Thrown at python/sglang/srt/multimodal/transport/cuda_ipc.py:44

    "_sglang_defer_cuda_ipc_feature_reconstruction"
)


def get_mm_feature_pool_size_per_worker(
    total_pool_size: int, tokenizer_worker_num: int
) -> int:
    """Split the CUDA IPC feature-pool budget without exceeding it.

    Each tokenizer worker owns a distinct CUDA allocation, even though all pools
    are created on ``base_gpu_id``.  Therefore a minimum per-worker allocation
    would make the aggregate HBM reservation larger than the configured budget.
    Keep the configured value as a hard per-node cap and leave at most
    ``tokenizer_worker_num - 1`` bytes unused when it is not evenly divisible.
    """
    if total_pool_size <= 0:
        raise ValueError("total_pool_size must be positive")
    if tokenizer_worker_num <= 0:
        raise ValueError("tokenizer_worker_num must be positive")

    return total_pool_size // tokenizer_worker_num


# Cache for pool-level IPC handles on the consumer side.
# Key: the pool CUDA IPC handle tuple. Value: opened UntypedStorage.
_pool_storage_cache: dict = {}
_pool_cache_lock = threading.Lock()


def _normalize_pool_cache_key(pool_handle, device_index: int) -> tuple[Any, ...]:
    normalized_handle = (
        pool_handle if isinstance(pool_handle, tuple) else tuple(pool_handle)
    )
    return (device_index, normalized_handle)


def _open_pooled_storage_uncached(pool_handle):

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure tokenizer worker count is set (>=1) before constructing the IPC transport
2. Fix --tokenizer-worker-num / dp-size configuration so at least one tokenizer worker exists
3. In tests, pass an explicit worker_num >= 1

Example fix

# before
get_mm_feature_pool_size_per_worker(pool_bytes, tokenizer_worker_num=0)
# after
get_mm_feature_pool_size_per_worker(pool_bytes, tokenizer_worker_num=4)

assert isinstance(tokenizer_worker_num, int) and tokenizer_worker_num >= 1

When it happens

Trigger: Initializing the CUDA IPC multimodal transport with tokenizer_worker_num <= 0, e.g. runtime parallel state not yet initialized returning 0, or a test passing 0 explicitly (test_rejects_invalid_budget_or_worker_count).

Common situations: Calling transport constructors before the scheduler/parallel runtime has set the tokenizer worker count; misconfigured --tokenizer-worker-num of 0; race during early startup.

Related errors

• total_pool_size must be positive
• memory_size must be positive
• consumer_count must be positive
• max_inflight_slices must be positive
• total_consumer_count must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/712e24e3bae040e3. Report an issue: GitHub.