{"record":{"id":"da3c4a85783d2827","repo":"sgl-project/sglang","slug":"prefill-only-disable-kv-cache-currently-requires-da3c4a","errorCode":null,"errorMessage":"--prefill-only-disable-kv-cache currently requires the FA prefill backend (fa3/fa4), but got prefill backend {prefill_backend!r}. Other prefill-only workloads and backends may be supported in a future change.","messagePattern":"--prefill-only-disable-kv-cache currently requires the FA prefill backend \\(fa3/fa4\\), but got prefill backend (.+?)\\. Other prefill-only workloads and backends may be supported in a future change\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/server_args.py","lineNumber":8238,"sourceCode":"        the default attention_backend if unset) and _handle_multi_item_scoring()\n        (which may further mutate it). The assertion below guards against\n        accidental call-site reordering: if the resolved attention_backend is\n        still None, backends haven't settled yet and the resolved (prefill,\n        decode) pair would be a stale (None, None).\n        \"\"\"\n        cfg = resolving_view(self)\n\n        if not cfg.prefill_only_disable_kv_cache:\n            return\n\n        assert resolved_view(self).attention_backend is not None, (\n            \"_handle_prefill_only_disable_kv_cache must run after \"\n            \"_handle_attention_backend_compatibility() so the prefill backend is resolved.\"\n        )\n\n        prefill_backend, _ = self._resolved_attention_backends()\n        if prefill_backend not in (\"fa3\", \"fa4\"):\n            raise ValueError(\n                \"--prefill-only-disable-kv-cache currently requires the FA prefill backend \"\n                f\"(fa3/fa4), but got prefill backend {prefill_backend!r}. Other prefill-only \"\n                \"workloads and backends may be supported in a future change.\"\n            )\n\n    def _handle_hicache_ratio_default(self):\n        \"\"\"Default the host/device ratio per host memory mode.\n\n        Runs before the dummy-model boundary: direct HostKVCache consumers\n        (unit fixtures, dummy-model launches) must never see a None ratio.\n        buffer_only stages in flight rather than retaining, so it needs only\n        enough to cover the write backlog plus parked prefetches.\n\n        A decode server keeps the ratio unset here: kv_cache_builder resolves\n        it against the retraction-backup backend (1.0 for host_pool, else 2.0).\n        \"\"\"\n        cfg = resolving_view(self)\n        if cfg.hicache_ratio is None and cfg.disaggregation_mode != \"decode\":","sourceCodeStart":8220,"sourceCodeEnd":8256,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/server_args.py#L8220-L8256","documentation":"The fa_skip_kv_cache fast path in SGLang is implemented only in the FlashAttention prefill backends. _handle_prefill_only_disable_kv_cache runs after attention backend resolution and raises if the resolved prefill backend is not fa3 or fa4.","triggerScenarios":"Launching with --prefill-only-disable-kv-cache while the resolved prefill attention backend is anything other than fa3/fa4 (e.g. flashinfer, triton, torch_native, or an AMD backend), typically via --attention-backend or a hardware default.","commonSituations":"Running on non-Hopper/Blackwell hardware where FA3 is unavailable, or a config that explicitly sets --attention-backend flashinfer for throughput, then adding the KV-cache-disabling flag.","solutions":["Set the prefill backend to FA3/FA4: --attention-backend fa3 (requires supported Hopper/Blackwell GPU + CUDA build)","Remove --prefill-only-disable-kv-cache if you must use another backend (e.g. flashinfer on other hardware)","On unsupported hardware, keep the real KV pool and use quantization instead"],"exampleFix":"# before\n--prefill-only-disable-kv-cache --attention-backend flashinfer\n# after\n--prefill-only-disable-kv-cache --attention-backend fa3","handlingStrategy":"validation","validationCode":"FA_BACKENDS = {\"fa3\", \"fa4\"}\nif want_disable_kv_cache and attention_backend not in FA_BACKENDS:\n    raise SystemExit(\"--prefill-only-disable-kv-cache needs --attention-backend fa3 on Hopper/Blackwell\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Confirm hardware support (H100/H200/B200) for FA3/FA4 before using the flag","Make attention backend an explicit flag rather than relying on auto-selection"],"tags":["sglang","attention-backend","flashattention","kv-cache","server-args"],"backgroundTag":"unsupported-backend","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}