{"record":{"id":"fa5406aa2fe458f3","repo":"vllm-project/vllm","slug":"use-replayssm-supports-prefix-caching-only-in-al","errorCode":null,"errorMessage":"--use-replayssm supports prefix caching only in align mode; pass --mamba-cache-mode align","messagePattern":"--use-replayssm supports prefix caching only in align mode; pass --mamba-cache-mode align","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2571,"sourceCode":"            raise ValueError(\n                \"--mamba-block-size can only be set with --enable-prefix-caching\"\n            )\n        return self\n\n    @model_validator(mode=\"after\")\n    def validate_mamba_cached_kernel(self) -> \"VllmConfig\":\n        if not self.cache_config.use_replayssm:\n            return self\n        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that\n        # opt in (supports_replayssm) build a consistent shape on both the layer\n        # and config paths. Reject others so the mamba page size cannot desync.\n        if self.model_config is not None and not self.model_config.supports_replayssm:\n            raise ValueError(\n                \"--use-replayssm is only supported for Nemotron-H models \"\n                f\"(got architecture {self.model_config.architecture!r})\"\n            )\n        if self.cache_config.mamba_cache_mode == \"all\":\n            raise ValueError(\n                \"--use-replayssm supports prefix caching only in align mode; \"\n                \"pass --mamba-cache-mode align\"\n            )\n        if self.num_speculative_tokens > 0:\n            raise ValueError(\"--use-replayssm does not support speculative decoding\")\n        if self.mamba_config.backend != MambaBackendEnum.TRITON:\n            raise ValueError(\"--use-replayssm requires --mamba-backend triton\")\n        if (\n            self.kv_transfer_config is not None\n            and self.kv_transfer_config.is_kv_transfer_instance\n        ):\n            raise ValueError(\n                \"--use-replayssm is incompatible with KV connectors \"\n                \"(P/D disaggregation, KV cache offload)\"\n            )\n        return self\n\n","sourceCodeStart":2553,"sourceCodeEnd":2589,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2553-L2589","documentation":"Raised by VllmConfig.validate_mamba_cached_kernel when the ReplaySSM feature (--use-replayssm) is enabled together with mamba_cache_mode == \"all\". ReplaySSM adds a 3-tensor ring to the mamba state, and prefix caching of that state is only coherent when the cache is aligned, so vLLM rejects the 'all' cache mode at config validation time.","triggerScenarios":"Starting vLLM with --use-replayssm while --mamba-cache-mode is set to (or defaults to) \"all\"; the check runs whenever cache_config.use_replayssm is true and cache_config.mamba_cache_mode == \"all\".","commonSituations":"Enabling ReplaySSM on a Nemotron-H model but copying mamba cache flags from a non-ReplaySSM config; using a launcher script that always passes --mamba-cache-mode all.","solutions":["Set --mamba-cache-mode align (or remove the flag if align is the default) and restart the server.","Verify the rest of the ReplaySSM constraints at the same time: Nemotron-H architecture only, no speculative decoding, --mamba-backend triton, and no KV transfer connector."],"exampleFix":"# before\nvllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm --mamba-cache-mode all\n# after\nvllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm --mamba-cache-mode align","handlingStrategy":"validation","validationCode":"from vllm.config import CacheConfigMode  # illustrative\ncfg = build_config(args)\nif cfg.cache_config.use_replayssm and cfg.cache_config.mamba_cache_mode == \"all\":\n    raise SystemExit(\"replayssm requires --mamba-cache-mode align\")","typeGuard":"def replayssm_config_valid(cfg) -> bool:\n    return not cfg.cache_config.use_replayssm or (\n        cfg.cache_config.mamba_cache_mode == \"align\"\n        and cfg.num_speculative_tokens == 0\n    )","tryCatchPattern":"try:\n    config = VllmConfig.from_cli(args)\nexcept ValueError as e:\n    if \"replayssm\" in str(e):\n        log_config_and_abort(e)  # surface the conflicting flags, do not retry\n    raise","preventionTips":["Validate the full replayssm flag set (model, cache mode, spec decode, backend, kv-transfer) in one preflight check before launching.","Keep replayssm launch profiles in a dedicated config file instead of composing flags ad hoc."],"tags":["vllm","config","mamba","replayssm","prefix-caching"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}