{"record":{"id":"dbd843026a0b21bc","repo":"sgl-project/sglang","slug":"total-pool-size-must-be-positive","errorCode":null,"errorMessage":"total_pool_size must be positive","messagePattern":"total_pool_size must be positive","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/transport/cuda_ipc.py","lineNumber":42,"sourceCode":"# model has computed the data-parallel assignment.\nDEFER_CUDA_IPC_FEATURE_RECONSTRUCTION_KEY = (\n    \"_sglang_defer_cuda_ipc_feature_reconstruction\"\n)\n\n\ndef get_mm_feature_pool_size_per_worker(\n    total_pool_size: int, tokenizer_worker_num: int\n) -> int:\n    \"\"\"Split the CUDA IPC feature-pool budget without exceeding it.\n\n    Each tokenizer worker owns a distinct CUDA allocation, even though all pools\n    are created on ``base_gpu_id``.  Therefore a minimum per-worker allocation\n    would make the aggregate HBM reservation larger than the configured budget.\n    Keep the configured value as a hard per-node cap and leave at most\n    ``tokenizer_worker_num - 1`` bytes unused when it is not evenly divisible.\n    \"\"\"\n    if total_pool_size <= 0:\n        raise ValueError(\"total_pool_size must be positive\")\n    if tokenizer_worker_num <= 0:\n        raise ValueError(\"tokenizer_worker_num must be positive\")\n\n    return total_pool_size // tokenizer_worker_num\n\n\n# Cache for pool-level IPC handles on the consumer side.\n# Key: the pool CUDA IPC handle tuple. Value: opened UntypedStorage.\n_pool_storage_cache: dict = {}\n_pool_cache_lock = threading.Lock()\n\n\ndef _normalize_pool_cache_key(pool_handle, device_index: int) -> tuple[Any, ...]:\n    normalized_handle = (\n        pool_handle if isinstance(pool_handle, tuple) else tuple(pool_handle)\n    )\n    return (device_index, normalized_handle)\n","sourceCodeStart":24,"sourceCodeEnd":60,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/transport/cuda_ipc.py#L24-L60","documentation":"get_mm_feature_pool_size_per_worker divides a configured CUDA IPC multimodal feature pool budget across tokenizer workers and requires the total byte budget to be strictly positive. Zero or negative sizes (misconfigured --mm-...-memory-budget, or an off-by-suffix like bytes vs MB producing 0 after int truncation) fail fast before any GPU allocation.","triggerScenarios":"Calling the CUDA IPC transport init (or its unit tests) with total_pool_size <= 0, e.g. budget string '0' parsed to 0 bytes, or negative value from config arithmetic.","commonSituations":"Setting the mm feature pool memory budget to 0 to 'disable' it instead of using the proper disable flag; parsing sizes in the wrong unit so integer division floors to 0; env var typos defaulting to 0.","solutions":["Set a positive budget (e.g. 1GB in bytes) in the server args for the mm feature pool","Check the unit parsing of the budget flag — ensure MB/GB conversion doesn't floor to 0","If you intended to disable the pool, use the supported disable mechanism rather than 0"],"exampleFix":"# before\n--cuda-ipc-mm-feature-pool-size 0\n# after\n--cuda-ipc-mm-feature-pool-size 1073741824","handlingStrategy":"validation","validationCode":"if total_pool_size is None or int(total_pool_size) <= 0:\n    raise ValueError('mm feature pool size must be a positive byte count')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never use 0 to disable a pool — use the documented disable flag","Add startup config assertions for memory budgets"],"tags":["cuda-ipc","memory-pool","config-validation","multimodal-transport"],"backgroundTag":"invalid-resource-size","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}