{"record":{"id":"16d22f35a78c6f93","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-is-not-supported-f","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache is not supported for {unsupported_pool_family}. Supported configurations today: plain MHA models on CUDA with the FA (fa3/fa4) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache, no context-parallel attention, no HiSparse, and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}.","messagePattern":"--prefill-only-disable-kv-cache is not supported for (.+?)\\. Supported configurations today: plain MHA models on CUDA with the FA \\(fa3/fa4\\) prefill backend, --is-embedding, --chunked-prefill-size=-1, --disable-radix-cache, no context-parallel attention, no HiSparse, and --kv-cache-dtype not in (.+?)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/kv_cache_configurator.py","lineNumber":751,"sourceCode":"        elif (\n            get_exec().kernel.attention_backend == \"ascend\" and not self.mambaish_config\n        ):\n            unsupported_pool_family = \"NPU/Ascend KV pool\"\n        elif self.use_mla_backend and self.is_hybrid_swa:\n            unsupported_pool_family = \"hybrid DSA/MLA-SWA KV pool\"\n        elif self.use_mla_backend and is_dsa_model:\n            unsupported_pool_family = \"DSA/MLA KV pool\"\n        elif self.use_mla_backend and not self.mambaish_config:\n            unsupported_pool_family = \"MLA KV pool\"\n        elif self.is_hybrid_swa:\n            unsupported_pool_family = \"SWA KV pool\"\n        elif self.mambaish_config:\n            unsupported_pool_family = \"hybrid linear/Mamba KV pool\"\n        elif is_float4_e2m1fn_x2(self.kv_cache_dtype):\n            unsupported_pool_family = \"FP4 MHA KV pool\"\n\n        if unsupported_pool_family is not None:\n            raise RuntimeError(\n                \"--prefill-only-disable-kv-cache is not supported for \"\n                f\"{unsupported_pool_family}. Supported configurations today: plain MHA \"\n                \"models on CUDA with the FA (fa3/fa4) prefill backend, --is-embedding, \"\n                \"--chunked-prefill-size=-1, --disable-radix-cache, no context-parallel \"\n                \"attention, no HiSparse, and --kv-cache-dtype not in {nvfp4, fp4_mx_block16}.\"\n            )\n\n    def _build_req_to_token_pool(self, *, max_num_reqs: int) -> ReqToTokenPool:\n        extra_max_context_len = get_req_to_token_extra_context_len()\n\n        if get_disagg().disaggregation_mode == \"decode\":\n            # Extra slots for pre-allocated requests\n            pre_alloc_size = get_disagg().disaggregation_decode_extra_slots\n            if self.mambaish_config:\n                req_to_token_pool = self._build_hybrid_mamba_decode_req_pool(\n                    max_num_reqs=max_num_reqs,\n                    extra_max_context_len=extra_max_context_len,\n                    pre_alloc_size=pre_alloc_size,","sourceCodeStart":733,"sourceCodeEnd":769,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/kv_cache_configurator.py#L733-L769","documentation":"A pre-pool validation in _validate_prefill_only_disable_kv_cache_pool_family: --prefill-only-disable-kv-cache rejects pool families it cannot back with a NoOp pool — currently hybrid linear/Mamba KV pools and FP4 (float4_e2m1fn_x2) MHA pools are named as unsupported.","triggerScenarios":"Set --prefill-only-disable-kv-cache on a model with a mambaish_config (hybrid linear/Mamba), or with --kv-cache-dtype nvfp4/fp4_mx_block16; the named unsupported_pool_family branch raises.","commonSituations":"Prefill-only serving (context embedding / ingest-only) attempted on hybrid SSM models or FP4-quantized KV cache.","solutions":["Remove --prefill-only-disable-kv-cache","Switch to a plain MHA model with auto/fp8 KV dtype if prefill-only mode is required"],"exampleFix":"# before\npython -m sglang.launch_server --model falcon-h1-... --prefill-only-disable-kv-cache\n# after\npython -m sglang.launch_server --model falcon-h1-...","handlingStrategy":"validation","validationCode":"if server_args.prefill_only_disable_kv_cache and (is_mambaish(model_config) or server_args.kv_cache_dtype in (\"nvfp4\", \"fp4_mx_block16\")):\n    raise SystemExit(\"prefill-only-disable-kv-cache unsupported for this pool family\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep a whitelist of model+dtype combos validated in CI for prefill-only mode"],"tags":["prefill-only","mamba","fp4-kv","kv-cache"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}