{"record":{"id":"a5d68dda36d645c8","repo":"vllm-project/vllm","slug":"mamba-block-size-can-only-be-set-with-enable-p","errorCode":null,"errorMessage":"--mamba-block-size can only be set with --enable-prefix-caching","messagePattern":"--mamba-block-size can only be set with --enable-prefix-caching","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2553,"sourceCode":"            and self.model_config.use_mla\n        ):\n            raise ValueError(\n                \"nvfp4 KV cache is not supported with MLA (Multi-head Latent \"\n                \"Attention) backends. Please use a different --kv-cache-dtype \"\n                \"(e.g., 'fp8' or 'auto') for MLA models such as DeepSeek.\"\n            )\n        return self\n\n    @model_validator(mode=\"after\")\n    def validate_mamba_block_size(self) -> \"VllmConfig\":\n        if self.model_config is None:\n            return self\n        mamba_block_size_is_set = (\n            self.cache_config.mamba_block_size is not None\n            and self.cache_config.mamba_block_size != self.model_config.max_model_len\n        )\n        if mamba_block_size_is_set and not self.cache_config.enable_prefix_caching:\n            raise ValueError(\n                \"--mamba-block-size can only be set with --enable-prefix-caching\"\n            )\n        return self\n\n    @model_validator(mode=\"after\")\n    def validate_mamba_cached_kernel(self) -> \"VllmConfig\":\n        if not self.cache_config.use_replayssm:\n            return self\n        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that\n        # opt in (supports_replayssm) build a consistent shape on both the layer\n        # and config paths. Reject others so the mamba page size cannot desync.\n        if self.model_config is not None and not self.model_config.supports_replayssm:\n            raise ValueError(\n                \"--use-replayssm is only supported for Nemotron-H models \"\n                f\"(got architecture {self.model_config.architecture!r})\"\n            )\n        if self.cache_config.mamba_cache_mode == \"all\":\n            raise ValueError(","sourceCodeStart":2535,"sourceCodeEnd":2571,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2535-L2571","documentation":"A non-default `--mamba-block-size` (set and different from max_model_len) only makes sense with the hybrid KV cache manager + prefix caching, because mamba state blocks are only managed that way. The `validate_mamba_block_size` model validator requires `--enable-prefix-caching` when mamba_block_size is overridden.","triggerScenarios":"Launching a Mamba/hybrid model with `--mamba-block-size N` (N != max_model_len) but without `--enable-prefix-caching`.","commonSituations":"Tuning mamba state page sizes for memory footprint on Mamba2/Nemotron-H models while prefix caching was disabled (e.g. by a baseline benchmarking script).","solutions":["Add `--enable-prefix-caching` to the launch command.","Or remove `--mamba-block-size` to use the default (block size follows max_model_len without prefix caching)."],"exampleFix":"# before\nvllm serve nvidia/Nemotron-H-8B-v1 --mamba-block-size 128\n\n# after\nvllm serve nvidia/Nemotron-H-8B-v1 --mamba-block-size 128 --enable-prefix-caching","handlingStrategy":"validation","validationCode":"if mamba_block_size is not None and mamba_block_size != max_model_len and not enable_prefix_caching:\n    enable_prefix_caching = True","typeGuard":null,"tryCatchPattern":"try:\n    LLM(mamba_block_size=128, ...)\nexcept ValueError as e:\n    if \"mamba-block-size\" in str(e):\n        LLM(mamba_block_size=128, enable_prefix_caching=True, ...)\n    else:\n        raise","preventionTips":["Treat --mamba-block-size as requiring --enable-prefix-caching in presets","Validate mamba flag pairs in launch linting"],"tags":["mamba","prefix-caching","block-size","startup-config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}