{"record":{"id":"680c3b20abf0e9f7","repo":"sgl-project/sglang","slug":"hicache-host-memory-mode-buffer-only-on-swa-mode","errorCode":null,"errorMessage":"--hicache-host-memory-mode buffer_only on SWA models requires an SWA host staging pool; the unified_kv layout keeps SWA as a device-only ring.","messagePattern":"--hicache-host-memory-mode buffer_only on SWA models requires an SWA host staging pool; the unified_kv layout keeps SWA as a device-only ring\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/buffer_mode/pipeline.py","lineNumber":168,"sourceCode":"def validate_buffer_only_stack(\n    sidecar_pool_specs: list, swa_component: Optional[SWAComponent]\n) -> None:\n    \"\"\"Post-assembly buffer-mode fences.\n\n    Sidecar pools (DSv4 compressed regions) and unified_kv SWA (device-only\n    ring, never offloaded) have no per-pool staging path yet.\n    \"\"\"\n    if sidecar_pool_specs:\n        raise ValueError(\n            \"--hicache-host-memory-mode buffer_only does not support \"\n            \"sidecar storage pools (DeepSeek-V4 compressed regions).\"\n        )\n    swa = swa_component\n    if swa is not None and swa._swa_kv_pool_host is None:\n        # Only reachable on SWA models with the unified_kv layout (SWA as\n        # a device-only ring): without a host pool the window can neither\n        # stage for writes nor fetch for load-backs.\n        raise ValueError(\n            \"--hicache-host-memory-mode buffer_only on SWA models \"\n            \"requires an SWA host staging pool; the unified_kv layout \"\n            \"keeps SWA as a device-only ring.\"\n        )\n    if swa is not None and swa._swa_kv_pool_host is not None:\n        # Below two windows the pool cannot hold a staging write AND the\n        # loads-priority reserve (_aux_loads_margin floors at one\n        # window), so every window-carrying intent would be dropped as\n        # oversize and SWA storage coverage would silently be zero.\n        window_tokens = swa.full_window_pages * swa._swa_kv_pool_host.page_size\n        if swa._swa_kv_pool_host.size < 2 * window_tokens:\n            raise ValueError(\n                \"--hicache-host-memory-mode buffer_only requires an SWA \"\n                f\"host pool of at least two trailing windows \"\n                f\"({2 * window_tokens} tokens; got \"\n                f\"{swa._swa_kv_pool_host.size}): one staging a write \"\n                \"while one stays reserved for prefetch window allocs.\"\n            )","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/buffer_mode/pipeline.py#L150-L186","documentation":"Thrown when buffer_only host memory mode is used with a sliding-window-attention (SWA) model that has no SWA host staging pool (unified_kv layout keeps SWA as a device-only ring). Without a host pool the SWA window can neither stage writes nor fetch load-backs, so the stack refuses to start.","triggerScenarios":"Server launch with --hicache-host-memory-mode buffer_only on an SWA model using the unified_kv layout where swa_component is non-None but _swa_kv_pool_host is None; validate_buffer_only_stack raises inside init_hicache.","commonSituations":"Running buffer_only mode on SWA/hybrid models (e.g. Gemma-style or other sliding window architectures) with the unified_kv pool layout; changing kv layout settings that disable SWA host offload.","solutions":["Disable buffer_only mode for SWA models (use default hicache host memory mode that creates the SWA host pool)","Configure the run so the SWA component gets a host staging pool (enable SWA offload / non-unified layout)","Use a non-SWA model if buffer_only is mandatory"],"exampleFix":"# before\n--hicache-host-memory-mode buffer_only   # SWA model, unified_kv layout\n# after\n# (drop the flag, or enable SWA host offload so _swa_kv_pool_host exists)","handlingStrategy":"validation","validationCode":"def swa_host_pool_ok(swa_comp):\n    return swa_comp is None or swa_comp._swa_kv_pool_host is not None\n# in launch config: if mode == 'buffer_only' and not swa_host_pool_ok(swa): use default mode","typeGuard":"def has_swa_host_pool(swa_comp) -> bool:\n    return swa_comp is not None and getattr(swa_comp, \"_swa_kv_pool_host\", None) is not None","tryCatchPattern":null,"preventionTips":["Gate buffer_only on model architecture (skip for SWA/unified_kv models)","Log the detected pool layout at startup so mismatches are visible early"],"tags":["hicache","buffer-only","swa","sliding-window","unified-kv","configuration"],"backgroundTag":"unsupported-feature-configuration","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}