vllm-project/vllm · error · ValueError

nvfp4 KV cache is not supported with MLA (Multi-head Latent

Error message

nvfp4 KV cache is not supported with MLA (Multi-head Latent Attention) backends. Please use a different --kv-cache-dtype (e.g., 'fp8' or 'auto') for MLA models such as DeepSeek.

What it means

NVFP4-quantized KV caches are only implemented for standard attention; MLA (Multi-head Latent Attention, used by DeepSeek-family models) has a single latent vector layout that the nvfp4 path does not support. The `validate_nvfp4_kv_cache_with_mla` model validator rejects any nvfp4-prefixed `--kv-cache-dtype` on an MLA model.

Source

Thrown at vllm/config/vllm.py:2537

            )

        # Mamba cache align-mode constraints
        if self.cache_config.mamba_cache_mode == "align":
            assert not self.scheduler_config.disable_chunked_mm_input, (
                "Chunked MM input is required because we need the flexibility "
                "to schedule a multiple of block_size tokens even if they are "
                "in the middle of a mm input"
            )

    @model_validator(mode="after")
    def validate_nvfp4_kv_cache_with_mla(self) -> "VllmConfig":
        if self.model_config is None:
            return self
        if (
            self.cache_config.cache_dtype.startswith("nvfp4")
            and self.model_config.use_mla
        ):
            raise ValueError(
                "nvfp4 KV cache is not supported with MLA (Multi-head Latent "
                "Attention) backends. Please use a different --kv-cache-dtype "
                "(e.g., 'fp8' or 'auto') for MLA models such as DeepSeek."
            )
        return self

    @model_validator(mode="after")
    def validate_mamba_block_size(self) -> "VllmConfig":
        if self.model_config is None:
            return self
        mamba_block_size_is_set = (
            self.cache_config.mamba_block_size is not None
            and self.cache_config.mamba_block_size != self.model_config.max_model_len
        )
        if mamba_block_size_is_set and not self.cache_config.enable_prefix_caching:
            raise ValueError(
                "--mamba-block-size can only be set with --enable-prefix-caching"
            )

View on GitHub (pinned to c794754062)

Solutions

1. Use a supported KV cache dtype for MLA: `--kv-cache-dtype fp8` or leave it as 'auto'.
2. Or switch to a non-MLA model if nvfp4 KV cache is a hard requirement.

Example fix

# before
vllm serve deepseek-ai/DeepSeek-R1 --kv-cache-dtype nvfp4

# after
vllm serve deepseek-ai/DeepSeek-R1 --kv-cache-dtype fp8

if model_config.use_mla and cache_dtype.startswith("nvfp4"):
    cache_dtype = "fp8"  # safe default for MLA models

try:
    LLM(model="deepseek-ai/DeepSeek-R1", kv_cache_dtype="nvfp4", ...)
except ValueError as e:
    if "nvfp4 KV cache is not supported with MLA" in str(e):
        LLM(model="deepseek-ai/DeepSeek-R1", kv_cache_dtype="fp8", ...)
    else:
        raise

When it happens

Trigger: Launching an MLA model (DeepSeek V2/V3/R1 and derivatives) with `--kv-cache-dtype nvfp4` (or another nvfp4* variant).

Common situations: Applying Blackwell-era KV compression flags validated on standard-attention models to DeepSeek deployments; copy-pasting GPU-memory-saving flag sets between models.

Related errors

• padded_n is not supported with TRTLLM 8x4 scale layout.
• chat request must contain at least one message
• asymmetric int8 activation quantization is unsupported on XP
• The optimized moe_wna16_gemm kernel is only available on CUD
• `a` must have at least 1 dimension.

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/a7a2d865c6fa98b7. Report an issue: GitHub.