{"record":{"id":"fa51025f25e3ec7b","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-does-not-currently-fa5102","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache does not currently support --kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate scale-factor buffers.","messagePattern":"--prefill-only-disable-kv-cache does not currently support --kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate scale-factor buffers\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":8168,"sourceCode":"\n        # This flag is intentionally scoped to embedding mode for now. Other\n        # prefill-only paths (for example scoring and MIS) can benefit from\n        # the same idea later, but some of them still stage K/V through the\n        # paged cache today.\n        if not cfg.is_embedding:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache currently requires --is-embedding. \"\n                \"Other prefill-only workloads may be supported in a future change once \"\n                \"their attention paths stop reading or writing the paged KV cache.\"\n            )\n        if cfg.kv_cache_dtype in (\"nvfp4\", \"fp4_mx_block16\"):\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because \"\n                \"the FP4 pool uses a separate allocation path.\"\n            )\n        if cfg.kv_cache_dtype == \"mxfp8\":\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate \"\n                \"scale-factor buffers.\"\n            )\n\n        # Structural preconditions for the FA backend's fa_skip_kv_cache path,\n        # which is the only embedding path that doesn't read or write the pool:\n        # - chunked_prefill_size == -1 keeps a request in a single forward,\n        #   so K/V never has to be reused across prefill chunks.\n        # - disable_radix_cache stops the prefix cache from indexing pool\n        #   slots that no longer hold real data.\n        if cfg.chunked_prefill_size != -1:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA \"\n                \"backend takes the fa_skip_kv_cache path; otherwise the pool would be touched \"\n                \"between prefill chunks.\"\n            )\n        if not cfg.disable_radix_cache:","sourceCodeStart":8150,"sourceCodeEnd":8186,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L8150-L8186","documentation":"--prefill-only-disable-kv-cache swaps in a no-op MHA KV pool, but the MXFP8 pool keeps separate scale-factor buffers alongside the data, which the no-op pool path does not manage. The validator therefore rejects --kv-cache-dtype=mxfp8 when KV-cache disabling is enabled.","triggerScenarios":"Launching with --prefill-only-disable-kv-cache and --kv-cache-dtype mxfp8 (cfg.kv_cache_dtype == \"mxfp8\").","commonSituations":"Enabling MXFP8 KV cache quantization (common on Blackwell for memory savings) on an embedding server that also disables the KV cache.","solutions":["Remove --kv-cache-dtype=mxfp8 when using --prefill-only-disable-kv-cache","If MXFP8 quantization is required, drop --prefill-only-disable-kv-cache","Watch upstream for MXFP8 scale-factor buffer support in the no-op pool path"],"exampleFix":"# before\n--is-embedding --prefill-only-disable-kv-cache --kv-cache-dtype mxfp8\n# after\n--is-embedding --prefill-only-disable-kv-cache","handlingStrategy":"validation","validationCode":"if want_disable_kv_cache and kv_cache_dtype == \"mxfp8\":\n    raise SystemExit(\"mxfp8 KV dtype is incompatible with --prefill-only-disable-kv-cache\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Centralize kv_cache_dtype selection in one config variable and assert compatibility once","Smoke-test full launch args against ServerArgs parsing in CI"],"tags":["sglang","kv-cache","mxfp8","quantization","server-args"],"backgroundTag":"unsupported-config-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}