{"record":{"id":"027bd39e63089ecb","repo":"sgl-project/sglang","slug":"expert-cache-and-staging-budgets-and-read-splits","errorCode":null,"errorMessage":"expert cache and staging budgets, and read splits, must be positive","messagePattern":"expert cache and staging budgets, and read splits, must be positive","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/expert_pack.py","lineNumber":184,"sourceCode":"    read_splits: int,\n    direct_io: bool,\n    stats_flush_interval: int,\n    stats_path: str | os.PathLike[str] | None,\n) -> None:\n    store.cache_vram_mib = int(cache_vram_mib)\n    store.cache_vram_reserve_mib = int(cache_vram_reserve_mib)\n    store.kernel_backend = \"custom\"\n    store.stage_slot_count = int(stage_slots)\n    store.read_splits = int(read_splits)\n    store.direct_io = bool(direct_io)\n    store.stats_flush_interval = int(stats_flush_interval)\n    if (\n        store.cache_vram_mib <= 0\n        or store.cache_vram_reserve_mib <= 0\n        or store.stage_slot_count <= 0\n        or store.read_splits <= 0\n    ):\n        raise ValueError(\n            \"expert cache and staging budgets, and read splits, must be positive\"\n        )\n    if store.stats_flush_interval < 0:\n        raise ValueError(\"expert-pack stats flush interval cannot be negative\")\n    if store.direct_io and not hasattr(os, \"O_DIRECT\"):\n        raise ValueError(\"expert-pack direct I/O is unavailable on this platform\")\n    open_flags = os.O_RDONLY | (os.O_DIRECT if store.direct_io else 0)\n    store._fd = os.open(store.path, open_flags)\n    store._lock = threading.RLock()\n    store._cache = None\n    store._cache_slots = []\n    store._key_to_slot = {}\n    store._key_frequency = {}\n    store._lru = OrderedDict()\n    store._staging = []\n    store._stage_events = []\n    store._stage_cursor = 0\n    store._transfer_stream = None","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/expert_pack.py#L166-L202","documentation":"ExpertPackStore._initialize_runtime_state requires cache_vram_mib, cache_vram_reserve_mib, stage_slot_count, and read_splits to all be > 0. These budgets control the VRAM LRU cache, staging slots, and split reads; zero/negative values would make the cache machinery unusable, so construction fails fast.","triggerScenarios":"Constructing ExpertPackStore with cache_vram_mib=0, cache_vram_reserve_mib<=0, stage_slot_count=0, or read_splits<=0 — commonly from defaults like 0 meaning 'auto' in older configs, or math that computes a budget of zero on small GPUs.","commonSituations":"Copy-pasted configs with cache_vram_mib=0 intending to disable caching (not supported); free-VRAM calculations that round to 0; negative values from misconfigured env vars or CLI overrides.","solutions":["Set explicit positive values, e.g. cache_vram_mib >= 64, cache_vram_reserve_mib >= 16, stage_slot_count >= 1, read_splits >= 1","If budgets are computed from free VRAM, clamp with max(1, computed) and log a warning","To effectively reduce caching, lower cache_vram_mib to a small positive number instead of 0"],"exampleFix":"# before\nstore = ExpertPackStore(p, cache_vram_mib=0, cache_vram_reserve_mib=0,\n                        stage_slot_count=0, read_splits=0)\n\n# after\nstore = ExpertPackStore(p, cache_vram_mib=512, cache_vram_reserve_mib=64,\n                        stage_slot_count=8, read_splits=4)","handlingStrategy":"validation","validationCode":"def budgets_ok(cache_vram_mib, cache_vram_reserve_mib, stage_slot_count, read_splits):\n    return (cache_vram_mib > 0 and cache_vram_reserve_mib > 0\n            and stage_slot_count > 0 and read_splits > 0)","typeGuard":null,"tryCatchPattern":"try:\n    store = ExpertPackStore(p, **cfg)\nexcept ValueError as e:\n    if \"must be positive\" in str(e):\n        cfg = {**cfg, \"cache_vram_mib\": 512, \"cache_vram_reserve_mib\": 64,\n               \"stage_slot_count\": 8, \"read_splits\": 4}\n        store = ExpertPackStore(p, **cfg)","preventionTips":["Clamp computed VRAM budgets with max(1, value)","Never use 0 to try to disable the cache; use a small positive budget"],"tags":["moe","expert-pack","configuration","vram-budget","argument-validation"],"backgroundTag":"invalid-configuration-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}