{"record":{"id":"784f2974b8937419","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-does-not-currently","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache does not currently support --kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because the FP4 pool uses a separate allocation path.","messagePattern":"--prefill-only-disable-kv-cache does not currently support --kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because the FP4 pool uses a separate allocation path\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":8162,"sourceCode":"        Backend resolution is checked separately by\n        _handle_prefill_only_disable_kv_cache after backends settle.\n        \"\"\"\n        cfg = resolving_view(self)\n        if not cfg.prefill_only_disable_kv_cache:\n            return\n\n        # This flag is intentionally scoped to embedding mode for now. Other\n        # prefill-only paths (for example scoring and MIS) can benefit from\n        # the same idea later, but some of them still stage K/V through the\n        # paged cache today.\n        if not cfg.is_embedding:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache currently requires --is-embedding. \"\n                \"Other prefill-only workloads may be supported in a future change once \"\n                \"their attention paths stop reading or writing the paged KV cache.\"\n            )\n        if cfg.kv_cache_dtype in (\"nvfp4\", \"fp4_mx_block16\"):\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because \"\n                \"the FP4 pool uses a separate allocation path.\"\n            )\n        if cfg.kv_cache_dtype == \"mxfp8\":\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate \"\n                \"scale-factor buffers.\"\n            )\n\n        # Structural preconditions for the FA backend's fa_skip_kv_cache path,\n        # which is the only embedding path that doesn't read or write the pool:\n        # - chunked_prefill_size == -1 keeps a request in a single forward,\n        #   so K/V never has to be reused across prefill chunks.\n        # - disable_radix_cache stops the prefix cache from indexing pool\n        #   slots that no longer hold real data.\n        if cfg.chunked_prefill_size != -1:","sourceCodeStart":8144,"sourceCodeEnd":8180,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L8144-L8180","documentation":"The no-op KV pool used by --prefill-only-disable-kv-cache assumes the standard token-to-KV allocation path. FP4 KV cache dtypes (nvfp4, fp4_mx_block16) allocate the pool through a separate FP4 path, so the combination is rejected up front in _validate_prefill_only_disable_kv_cache_args.","triggerScenarios":"Launching with --prefill-only-disable-kv-cache together with --kv-cache-dtype nvfp4 or --kv-cache-dtype fp4_mx_block16 (cfg.kv_cache_dtype in (\"nvfp4\",\"fp4_mx_block16\")).","commonSituations":"A user enables FP4 KV cache quantization to save memory and separately enables KV-cache disabling for embedding serving, not realizing the FP4 pool has its own allocator.","solutions":["Remove --kv-cache-dtype=nvfp4 / fp4_mx_block16 (fall back to the default dtype) when using --prefill-only-disable-kv-cache","If FP4 KV cache is required, drop --prefill-only-disable-kv-cache and rely on quantization for memory savings","Wait for upstream support of the FP4 allocation path with the no-op pool"],"exampleFix":"# before\n--is-embedding --prefill-only-disable-kv-cache --kv-cache-dtype nvfp4\n# after\n--is-embedding --prefill-only-disable-kv-cache","handlingStrategy":"validation","validationCode":"FP4_DTYPES = {\"nvfp4\", \"fp4_mx_block16\"}\nif want_disable_kv_cache and kv_cache_dtype in FP4_DTYPES:\n    raise SystemExit(\"pick one: --prefill-only-disable-kv-cache or FP4 kv-cache-dtype\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep a compatibility matrix of kv-cache-dtype vs memory flags in your serving config repo","Never stack KV memory optimizations (quantized pool + disabled pool) without checking the validator"],"tags":["sglang","kv-cache","fp4","quantization","server-args"],"backgroundTag":"unsupported-config-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}