{"record":{"id":"71fe9f867cede1c6","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-requires-chunked","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA backend takes the fa_skip_kv_cache path; otherwise the pool would be touched between prefill chunks.","messagePattern":"--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA backend takes the fa_skip_kv_cache path; otherwise the pool would be touched between prefill chunks\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":8181,"sourceCode":"                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=nvfp4 or --kv-cache-dtype=fp4_mx_block16 because \"\n                \"the FP4 pool uses a separate allocation path.\"\n            )\n        if cfg.kv_cache_dtype == \"mxfp8\":\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache does not currently support \"\n                \"--kv-cache-dtype=mxfp8 because the MXFP8 pool stores separate \"\n                \"scale-factor buffers.\"\n            )\n\n        # Structural preconditions for the FA backend's fa_skip_kv_cache path,\n        # which is the only embedding path that doesn't read or write the pool:\n        # - chunked_prefill_size == -1 keeps a request in a single forward,\n        #   so K/V never has to be reused across prefill chunks.\n        # - disable_radix_cache stops the prefix cache from indexing pool\n        #   slots that no longer hold real data.\n        if cfg.chunked_prefill_size != -1:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache requires --chunked-prefill-size=-1 so the FA \"\n                \"backend takes the fa_skip_kv_cache path; otherwise the pool would be touched \"\n                \"between prefill chunks.\"\n            )\n        if not cfg.disable_radix_cache:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache requires --disable-radix-cache because the \"\n                \"radix cache indexes KV pool slots that no longer hold real data.\"\n            )\n\n        # Context-parallel prefill stages K/V through cp_allgather_and_save_kv_cache,\n        # which writes to the pool via set_kv_buffer. NoOpMHATokenToKVPool intentionally\n        # raises on writes, so the engine would boot fine but fail on the first request.\n        if self._resolved().attn_cp_size > 1:\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache is incompatible with --attn-cp-size > 1: \"\n                \"the context-parallel attention path writes K/V to the pool via set_kv_buffer, \"\n                \"which the no-op pool intentionally rejects.\"","sourceCodeStart":8163,"sourceCodeEnd":8199,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L8163-L8199","documentation":"The FA backend only takes its fa_skip_kv_cache path (which never touches the pool) when a prefill fits in a single forward. chunked_prefill_size == -1 guarantees that; any other value means K/V must be reused across prefill chunks, so --prefill-only-disable-kv-cache requires it.","triggerScenarios":"Launching with --prefill-only-disable-kv-cache while cfg.chunked_prefill_size != -1 (default chunked prefill, or an explicit positive value).","commonSituations":"A user copies a general serving config (which sets --chunked-prefill-size 8192 or relies on the default) and adds --prefill-only-disable-kv-cache for embedding serving.","solutions":["Set --chunked-prefill-size -1 in the launch command","Check server logs/scripts for a chunked-prefill-size default injected by a profile or launch wrapper and override it","Ensure the model's context length fits in one forward, since -1 disables chunking"],"exampleFix":"# before\n--prefill-only-disable-kv-cache --chunked-prefill-size 8192\n# after\n--prefill-only-disable-kv-cache --chunked-prefill-size -1","handlingStrategy":"validation","validationCode":"if want_disable_kv_cache and chunked_prefill_size != -1:\n    raise SystemExit(\"set --chunked-prefill-size -1 for --prefill-only-disable-kv-cache\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["When copying serving configs, strip chunked-prefill overrides for embedding workloads","Verify GPU memory fits an unchunked forward for your max sequence length"],"tags":["sglang","chunked-prefill","kv-cache","flashattention","server-args"],"backgroundTag":"invalid-flag-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}