{"record":{"id":"cb2a458448d957cf","repo":"sgl-project/sglang","slug":"speculative-decoding-with-enable-unified-memory","errorCode":null,"errorMessage":"Speculative decoding with --enable-unified-memory is only supported for hybrid-Mamba targets; the unified hybrid-SWA pool's draft sizing (virtual-id space) is not wired yet.","messagePattern":"Speculative decoding with --enable-unified-memory is only supported for hybrid-Mamba targets; the unified hybrid-SWA pool's draft sizing \\(virtual-id space\\) is not wired yet\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/kv_cache_configurator.py","lineNumber":441,"sourceCode":"            return _InitializedPools(\n                req_to_token_pool=bundle.req_to_token_pool,\n                token_to_kv_pool=bundle.token_to_kv_pool,\n                token_to_kv_pool_allocator=bundle.token_to_kv_pool_allocator,\n                unified_memory_pool=bundle.unified_memory_pool,\n            )\n\n        # The unified allocator hands out VIRTUAL token ids from the whole\n        # virtual space (> max_total_num_tokens); the direct-indexed draft\n        # pool must be sized by that space.\n        draft_virtual_id_space: Optional[int] = None\n        if self.is_draft_worker and token_to_kv_pool_allocator is not None:\n            from sglang.srt.mem_cache.multi_ended_allocator import (\n                UnifiedMambaTokenToKVPoolAllocator,\n                UnifiedSWATokenToKVPoolAllocator,\n            )\n\n            if isinstance(token_to_kv_pool_allocator, UnifiedSWATokenToKVPoolAllocator):\n                raise ValueError(\n                    \"Speculative decoding with --enable-unified-memory is only \"\n                    \"supported for hybrid-Mamba targets; the unified hybrid-SWA \"\n                    \"pool's draft sizing (virtual-id space) is not wired yet.\"\n                )\n            if isinstance(\n                token_to_kv_pool_allocator, UnifiedMambaTokenToKVPoolAllocator\n            ):\n                draft_virtual_id_space = token_to_kv_pool_allocator.size_full\n                assert draft_virtual_id_space >= sizes.max_total_num_tokens, (\n                    \"unified allocator virtual space smaller than the token \"\n                    f\"budget: size_full={draft_virtual_id_space} < \"\n                    f\"max_total_num_tokens={sizes.max_total_num_tokens}\"\n                )\n                # Round UP to page alignment (paged draft backends view the\n                # pool as (-1, page_size, H, D); size_full is not aligned).\n                page = max(int(self.pool_page_size or 1), 1)\n                draft_virtual_id_space = (\n                    (draft_virtual_id_space + page - 1) // page * page","sourceCodeStart":423,"sourceCodeEnd":459,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/kv_cache_configurator.py#L423-L459","documentation":"When speculative decoding is enabled together with --enable-unified-memory, the resulting allocator is checked: if it is a UnifiedSWATokenToKVPoolAllocator the run is rejected, because the unified hybrid-SWA pool's draft sizing (virtual-id space) is not wired up yet. Only the hybrid-Mamba unified allocator supports a draft worker.","triggerScenarios":"Launch with --enable-unified-memory plus a speculative decoding algorithm on a hybrid-SWA model; the allocator isinstance check in _init_pools fails.","commonSituations":"Enabling spec decode (e.g. EAGLE-style) on a hybrid-SWA model that uses the unified memory pool.","solutions":["Disable speculative decoding for this model","Drop --enable-unified-memory","Use a hybrid-Mamba model where the unified allocator supports spec decode"],"exampleFix":"# before\npython -m sglang.launch_server --model hybrid-swa --enable-unified-memory --speculative-algorithm EAGLE ...\n# after\npython -m sglang.launch_server --model hybrid-swa --enable-unified-memory ...","handlingStrategy":"validation","validationCode":"if server_args.enable_unified_memory and server_args.speculative_algorithm and is_hybrid_swa(model_config):\n    raise SystemExit(\"spec decode + unified-memory unsupported on hybrid-SWA; drop one\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Track which spec-decode paths are wired per pool type before enabling both"],"tags":["speculative-decoding","unified-memory","hybrid-swa","kv-cache"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}