{"record":{"id":"67bc7c91ced148e6","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-expected-noopmhato","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache expected NoOpMHATokenToKVPool but the runtime pool is {type(token_to_kv_pool).__name__}. This pool family is not yet supported by --prefill-only-disable-kv-cache. Supported configurations today: plain MHA models on CUDA with the FA (fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache, no context-parallel attention, no HiSparse, and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}.","messagePattern":"--prefill-only-disable-kv-cache expected NoOpMHATokenToKVPool but the runtime pool is (.+?)\\. This pool family is not yet supported by --prefill-only-disable-kv-cache\\. Supported configurations today: plain MHA models on CUDA with the FA \\(fa3/fa4\\) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache, no context-parallel attention, no HiSparse, and --kv-cache-dtype not in (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/kv_cache_configurator.py","lineNumber":532,"sourceCode":"            )\n\n        token_to_kv_pool_allocator = self._build_token_to_kv_pool_allocator(\n            sizes=sizes,\n            token_to_kv_pool=token_to_kv_pool,\n            is_dsv4_model=is_dsv4_model,\n            req_to_token_pool=req_to_token_pool,\n            token_to_kv_pool_allocator=token_to_kv_pool_allocator,\n        )\n\n        # Defensive check: the explicit validation above should reject known\n        # unsupported pool families before allocation. Keep this guard here so\n        # future pool-selection refactors fail at boot instead of on first use.\n        if (\n            get_schedule().prefill_only_disable_kv_cache\n            and not self.is_draft_worker\n            and not isinstance(token_to_kv_pool, NoOpMHATokenToKVPool)\n        ):\n            raise RuntimeError(\n                \"--prefill-only-disable-kv-cache expected NoOpMHATokenToKVPool but the \"\n                f\"runtime pool is {type(token_to_kv_pool).__name__}. This pool \"\n                \"family is not yet supported by --prefill-only-disable-kv-cache. \"\n                \"Supported configurations today: plain MHA models on CUDA with the FA \"\n                \"(fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, \"\n                \"--disable-radix-cache, no context-parallel attention, no HiSparse, \"\n                \"and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}.\"\n            )\n        return _InitializedPools(\n            req_to_token_pool=req_to_token_pool,\n            token_to_kv_pool=token_to_kv_pool,\n            token_to_kv_pool_allocator=token_to_kv_pool_allocator,\n        )\n\n    def _init_unified_mamba_pools(\n        self, *, max_num_reqs: int, max_total_num_tokens: int\n    ) -> UnifiedPoolBundle:\n        \"\"\"Build the shared-KV-pool stack for a hybrid-Mamba model:","sourceCodeStart":514,"sourceCodeEnd":550,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/kv_cache_configurator.py#L514-L550","documentation":"A post-boot invariant check: when --prefill-only-disable-kv-cache is set (and this is not the draft worker), the runtime KV pool must be a NoOpMHATokenToKVPool. Any other pool class means the configuration quietly produced a real/unsupported pool, so the server fails instead of misbehaving at first request.","triggerScenarios":"Enable --prefill-only-disable-kv-cache on a config that doesn't reduce to a plain MHA CUDA FA3/FA4 pool: hybrid/Mamba models, quantized KV dtype (nvfp4/fp4_mx_block16), radix cache on, chunked prefill enabled, CP attention, HiSparse, etc.","commonSituations":"Using the prefill-only (no KV cache) mode on an unsupported model/backend, or a pool-selection refactor changing which pool class gets built.","solutions":["Check the message for the actual pool class and remove the unsupported option (e.g. drop --kv-cache-dtype nvfp4, disable CP/HiSparse)","Only use --prefill-only-disable-kv-cache with plain MHA models on CUDA + FA (fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache","If you believe the config should be supported, report a bug — the pool family list is the source of truth"],"exampleFix":"# before\npython -m sglang.launch_server --model qwen-... --prefill-only-disable-kv-cache --kv-cache-dtype nvfp4\n# after\npython -m sglang.launch_server --model qwen-... --prefill-only-disable-kv-cache --kv-cache-dtype auto","handlingStrategy":"validation","validationCode":"sa = server_args\nok = (sa.prefill_only_disable_kv_cache and sa.device == \"cuda\"\n      and sa.attention_backend in (\"fa3\", \"fa4\")\n      and sa.chunked_prefill_size == -1 and sa.disable_radix_cache\n      and sa.kv_cache_dtype not in (\"nvfp4\", \"fp4_mx_block16\")\n      and not sa.enable_context_parallel and not getattr(sa, \"enable_hiparse\", False))\nassert ok, \"--prefill-only-disable-kv-cache config unsupported\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Centralize the supported-config predicate in your deployment tooling","Log the built pool class at boot so regressions surface early"],"tags":["prefill-only","kv-cache","pool-family","boot-config"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}