sgl-project/sglang · error · ValueError

expert cache and staging budgets, and read splits, must be p

Error message

expert cache and staging budgets, and read splits, must be positive

What it means

ExpertPackStore._initialize_runtime_state requires cache_vram_mib, cache_vram_reserve_mib, stage_slot_count, and read_splits to all be > 0. These budgets control the VRAM LRU cache, staging slots, and split reads; zero/negative values would make the cache machinery unusable, so construction fails fast.

Source

Thrown at python/sglang/srt/layers/moe/expert_pack.py:184

    read_splits: int,
    direct_io: bool,
    stats_flush_interval: int,
    stats_path: str | os.PathLike[str] | None,
) -> None:
    store.cache_vram_mib = int(cache_vram_mib)
    store.cache_vram_reserve_mib = int(cache_vram_reserve_mib)
    store.kernel_backend = "custom"
    store.stage_slot_count = int(stage_slots)
    store.read_splits = int(read_splits)
    store.direct_io = bool(direct_io)
    store.stats_flush_interval = int(stats_flush_interval)
    if (
        store.cache_vram_mib <= 0
        or store.cache_vram_reserve_mib <= 0
        or store.stage_slot_count <= 0
        or store.read_splits <= 0
    ):
        raise ValueError(
            "expert cache and staging budgets, and read splits, must be positive"
        )
    if store.stats_flush_interval < 0:
        raise ValueError("expert-pack stats flush interval cannot be negative")
    if store.direct_io and not hasattr(os, "O_DIRECT"):
        raise ValueError("expert-pack direct I/O is unavailable on this platform")
    open_flags = os.O_RDONLY | (os.O_DIRECT if store.direct_io else 0)
    store._fd = os.open(store.path, open_flags)
    store._lock = threading.RLock()
    store._cache = None
    store._cache_slots = []
    store._key_to_slot = {}
    store._key_frequency = {}
    store._lru = OrderedDict()
    store._staging = []
    store._stage_events = []
    store._stage_cursor = 0
    store._transfer_stream = None

View on GitHub (pinned to 0132848349)

Solutions

Set explicit positive values, e.g. cache_vram_mib >= 64, cache_vram_reserve_mib >= 16, stage_slot_count >= 1, read_splits >= 1
If budgets are computed from free VRAM, clamp with max(1, computed) and log a warning
To effectively reduce caching, lower cache_vram_mib to a small positive number instead of 0

Example fix

# before
store = ExpertPackStore(p, cache_vram_mib=0, cache_vram_reserve_mib=0,
                        stage_slot_count=0, read_splits=0)

# after
store = ExpertPackStore(p, cache_vram_mib=512, cache_vram_reserve_mib=64,
                        stage_slot_count=8, read_splits=4)

Defensive patterns

Strategy: validation

Validate before calling

def budgets_ok(cache_vram_mib, cache_vram_reserve_mib, stage_slot_count, read_splits):
    return (cache_vram_mib > 0 and cache_vram_reserve_mib > 0
            and stage_slot_count > 0 and read_splits > 0)

Try / catch

try:
    store = ExpertPackStore(p, **cfg)
except ValueError as e:
    if "must be positive" in str(e):
        cfg = {**cfg, "cache_vram_mib": 512, "cache_vram_reserve_mib": 64,
               "stage_slot_count": 8, "read_splits": 4}
        store = ExpertPackStore(p, **cfg)

Prevention

Clamp computed VRAM budgets with max(1, value)
Never use 0 to try to disable the cache; use a small positive budget

When it happens

Trigger: Constructing ExpertPackStore with cache_vram_mib=0, cache_vram_reserve_mib<=0, stage_slot_count=0, or read_splits<=0 — commonly from defaults like 0 meaning 'auto' in older configs, or math that computes a budget of zero on small GPUs.

Common situations: Copy-pasted configs with cache_vram_mib=0 intending to disable caching (not supported); free-VRAM calculations that round to 0; negative values from misconfigured env vars or CLI overrides.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/027bd39e63089ecb. Report an issue: GitHub.