{"record":{"id":"eb1529fbdaa68b5a","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-currently-requires","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache currently requires --is-embedding. Other prefill-only workloads may be supported in a future change once their attention paths stop reading or writing the paged KV cache.","messagePattern":"--prefill-only-disable-kv-cache currently requires --is-embedding\\. Other prefill-only workloads may be supported in a future change once their attention paths stop reading or writing the paged KV cache\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":8156,"sourceCode":"\n        run_post_process_pass(self, _pipeline_parallel_overlap_disable)\n\n    def _validate_prefill_only_disable_kv_cache_args(self):\n        \"\"\"Validate --prefill-only-disable-kv-cache flag/precondition constraints.\n\n        Backend resolution is checked separately by\n        _handle_prefill_only_disable_kv_cache after backends settle.\n        \"\"\"\n        cfg = resolving_view(self)\n        if not cfg.prefill_only_disable_kv_cache:\n            return\n\n        # This flag is intentionally scoped to embedding mode for now. Other\n        # prefill-only paths (for example scoring and MIS) can benefit from\n        # the same idea later, but some of them still stage K/V through the\n        # paged cache today.\n        if not cfg.is_embedding:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache currently requires --is-embedding. \"\n                \"Other prefill-only workloads may be supported in a future change once \"\n                \"their attention paths stop reading or writing the paged KV cache.\"\n            )\n        if cfg.kv_cache_dtype in (\"nvfp4\", \"fp4_mx_block16\"):\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because \"\n                \"the FP4 pool uses a separate allocation path.\"\n            )\n        if cfg.kv_cache_dtype == \"mxfp8\":\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate \"\n                \"scale-factor buffers.\"\n            )\n\n        # Structural preconditions for the FA backend's fa_skip_kv_cache path,","sourceCodeStart":8138,"sourceCodeEnd":8174,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L8138-L8174","documentation":"SGLang's --prefill-only-disable-kv-cache flag replaces the paged KV cache with a no-op MHA pool, and that path is currently gated to embedding serving (--is-embedding). Other prefill-only workloads (scoring, MIS) still stage K/V through the paged cache, so enabling the flag without embedding mode is rejected during server-args resolution.","triggerScenarios":"Launching the server with --prefill-only-disable-kv-cache but without --is-embedding (or without using an embedding model that auto-sets it), causing _validate_prefill_only_disable_kv_cache_args to raise during the resolution pipeline.","commonSituations":"A user tries to shave memory off a non-embedding prefill workload (e.g. reranking/scoring) by disabling the KV cache, assuming the flag is generic; or an embedding workload where --is-embedding was not passed/auto-detected.","solutions":["Add --is-embedding to the launch command (or use an embedding model so it is auto-set)","If the workload is not embedding (scoring, MIS), remove --prefill-only-disable-kv-cache until support lands","Track the upstream change that makes other prefill-only attention paths pool-free before re-enabling"],"exampleFix":"# before\npython -m sglang.launch_server --model qwen3-reranker --prefill-only-disable-kv-cache\n# after\npython -m sglang.launch_server --model Qwen3-Embedding --is-embedding --prefill-only-disable-kv-cache","handlingStrategy":"validation","validationCode":"from sglang.srt.server_args import prepare_server_args\nargs = [\"--model\", MODEL, *user_flags]\nif \"--prefill-only-disable-kv-cache\" in args and \"--is-embedding\" not in args and not is_embedding_model(MODEL):\n    raise SystemExit(\"--prefill-only-disable-kv-cache requires --is-embedding\")\nserver_args = prepare_server_args(args)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Treat --prefill-only-disable-kv-cache as an embedding-only flag in your launch templates","Run a dry parse of ServerArgs before starting the scheduler so config errors surface early"],"tags":["sglang","kv-cache","prefill","embedding","server-args"],"backgroundTag":"invalid-flag-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}