{"record":{"id":"1bf39668d7bb5814","repo":"vllm-project/vllm","slug":"fast-prefill-optimization-for-kv-sharing-is-not-co","errorCode":null,"errorMessage":"Fast prefill optimization for KV sharing is not compatible with EAGLE as EAGLE requires correct logits for all tokens while fast prefill gives incorrect logits for prompt tokens.","messagePattern":"Fast prefill optimization for KV sharing is not compatible with EAGLE as EAGLE requires correct logits for all tokens while fast prefill gives incorrect logits for prompt tokens\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":1550,"sourceCode":"                logger.info_once(\"Cudagraph is disabled under eager mode\")\n                self.compilation_config.cudagraph_mode = CUDAGraphMode.NONE\n                # override related settings when enforce eager\n                self.compilation_config.max_cudagraph_capture_size = 0\n                self.compilation_config.cudagraph_capture_sizes = []\n            else:\n                self.compilation_config.cudagraph_num_of_warmups = 1\n\n            self._set_cudagraph_sizes()\n\n        else:\n            self.compilation_config.cudagraph_mode = CUDAGraphMode.NONE\n\n        if self.cache_config.kv_sharing_fast_prefill:\n            if (\n                self.speculative_config is not None\n                and self.speculative_config.use_eagle()\n            ):\n                raise ValueError(\n                    \"Fast prefill optimization for KV sharing is not \"\n                    \"compatible with EAGLE as EAGLE requires correct logits \"\n                    \"for all tokens while fast prefill gives incorrect logits \"\n                    \"for prompt tokens.\"\n                )\n\n            logger.warning_once(\n                \"--kv-sharing-fast-prefill requires changes on model side for \"\n                \"correctness and to realize prefill savings.\"\n            )\n\n        if (\n            self.model_config\n            and self.model_config.architecture == \"WhisperForConditionalGeneration\"\n            and os.environ.get(\"VLLM_WORKER_MULTIPROC_METHOD\") != \"spawn\"\n        ):\n            logger.warning_once(\n                \"Whisper is known to have issues with \"","sourceCodeStart":1532,"sourceCodeEnd":1568,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L1532-L1568","documentation":"`--kv-sharing-fast-prefill` skips full attention computation for prompt tokens whose KV is shared, which produces incorrect logits for those tokens. EAGLE speculative decoding needs correct logits for every token to build its draft distribution, so the combination is rejected at validation time.","triggerScenarios":"Launching with `--kv-sharing-fast-prefill` and a speculative config that uses EAGLE (`speculative_config.use_eagle()` true, incl. MTP variants).","commonSituations":"Users of models with cross-layer KV sharing (e.g. MLA/CLV models like some Qwen3/Gemma variants) enabling the fast-prefill latency optimization while also running EAGLE drafts.","solutions":["Remove `--kv-sharing-fast-prefill` when using EAGLE speculative decoding.","Or use a non-EAGLE spec-decode method (e.g. ngram GPU) that tolerates incorrect prompt logits.","Or disable speculative decoding entirely for this deployment."],"exampleFix":"# before\nvllm serve model --kv-sharing-fast-prefill \\\n  --speculative-config '{\"method\":\"eagle\",...}'\n\n# after\nvllm serve model \\\n  --speculative-config '{\"method\":\"eagle\",...}'","handlingStrategy":"validation","validationCode":"if kv_sharing_fast_prefill and spec_config and spec_config.get(\"method\", \"\").lower().startswith((\"eagle\", \"mtp\")):\n    kv_sharing_fast_prefill = False","typeGuard":null,"tryCatchPattern":"try:\n    LLM(**args)\nexcept ValueError as e:\n    if \"Fast prefill optimization for KV sharing\" in str(e):\n        args[\"kv_sharing_fast_prefill\"] = False\n    else:\n        raise","preventionTips":["Document that kv-sharing-fast-prefill trades correct prompt logits for speed","Pre-check spec-decode method before enabling prefill optimizations"],"tags":["kv-cache","kv-sharing","eagle","speculative-decoding"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}