{"record":{"id":"a7a2d865c6fa98b7","repo":"vllm-project/vllm","slug":"nvfp4-kv-cache-is-not-supported-with-mla-multi-he","errorCode":null,"errorMessage":"nvfp4 KV cache is not supported with MLA (Multi-head Latent Attention) backends. Please use a different --kv-cache-dtype (e.g., 'fp8' or 'auto') for MLA models such as DeepSeek.","messagePattern":"nvfp4 KV cache is not supported with MLA \\(Multi-head Latent Attention\\) backends\\. Please use a different --kv-cache-dtype \\(e\\.g\\., 'fp8' or 'auto'\\) for MLA models such as DeepSeek\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2537,"sourceCode":"            )\n\n        # Mamba cache align-mode constraints\n        if self.cache_config.mamba_cache_mode == \"align\":\n            assert not self.scheduler_config.disable_chunked_mm_input, (\n                \"Chunked MM input is required because we need the flexibility \"\n                \"to schedule a multiple of block_size tokens even if they are \"\n                \"in the middle of a mm input\"\n            )\n\n    @model_validator(mode=\"after\")\n    def validate_nvfp4_kv_cache_with_mla(self) -> \"VllmConfig\":\n        if self.model_config is None:\n            return self\n        if (\n            self.cache_config.cache_dtype.startswith(\"nvfp4\")\n            and self.model_config.use_mla\n        ):\n            raise ValueError(\n                \"nvfp4 KV cache is not supported with MLA (Multi-head Latent \"\n                \"Attention) backends. Please use a different --kv-cache-dtype \"\n                \"(e.g., 'fp8' or 'auto') for MLA models such as DeepSeek.\"\n            )\n        return self\n\n    @model_validator(mode=\"after\")\n    def validate_mamba_block_size(self) -> \"VllmConfig\":\n        if self.model_config is None:\n            return self\n        mamba_block_size_is_set = (\n            self.cache_config.mamba_block_size is not None\n            and self.cache_config.mamba_block_size != self.model_config.max_model_len\n        )\n        if mamba_block_size_is_set and not self.cache_config.enable_prefix_caching:\n            raise ValueError(\n                \"--mamba-block-size can only be set with --enable-prefix-caching\"\n            )","sourceCodeStart":2519,"sourceCodeEnd":2555,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2519-L2555","documentation":"NVFP4-quantized KV caches are only implemented for standard attention; MLA (Multi-head Latent Attention, used by DeepSeek-family models) has a single latent vector layout that the nvfp4 path does not support. The `validate_nvfp4_kv_cache_with_mla` model validator rejects any nvfp4-prefixed `--kv-cache-dtype` on an MLA model.","triggerScenarios":"Launching an MLA model (DeepSeek V2/V3/R1 and derivatives) with `--kv-cache-dtype nvfp4` (or another nvfp4* variant).","commonSituations":"Applying Blackwell-era KV compression flags validated on standard-attention models to DeepSeek deployments; copy-pasting GPU-memory-saving flag sets between models.","solutions":["Use a supported KV cache dtype for MLA: `--kv-cache-dtype fp8` or leave it as 'auto'.","Or switch to a non-MLA model if nvfp4 KV cache is a hard requirement."],"exampleFix":"# before\nvllm serve deepseek-ai/DeepSeek-R1 --kv-cache-dtype nvfp4\n\n# after\nvllm serve deepseek-ai/DeepSeek-R1 --kv-cache-dtype fp8","handlingStrategy":"validation","validationCode":"if model_config.use_mla and cache_dtype.startswith(\"nvfp4\"):\n    cache_dtype = \"fp8\"  # safe default for MLA models","typeGuard":null,"tryCatchPattern":"try:\n    LLM(model=\"deepseek-ai/DeepSeek-R1\", kv_cache_dtype=\"nvfp4\", ...)\nexcept ValueError as e:\n    if \"nvfp4 KV cache is not supported with MLA\" in str(e):\n        LLM(model=\"deepseek-ai/DeepSeek-R1\", kv_cache_dtype=\"fp8\", ...)\n    else:\n        raise","preventionTips":["Restrict nvfp4 kv-cache-dtype to non-MLA models","Per-model flag presets instead of one global flag set"],"tags":["kv-cache","nvfp4","mla","deepseek","quantization"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}