{"record":{"id":"1b37c46a30753abd","repo":"vllm-project/vllm","slug":"use-replayssm-is-only-supported-for-nemotron-h-m","errorCode":null,"errorMessage":"--use-replayssm is only supported for Nemotron-H models (got architecture {self.model_config.architecture!r})","messagePattern":"--use-replayssm is only supported for Nemotron-H models \\(got architecture (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2566,"sourceCode":"        mamba_block_size_is_set = (\n            self.cache_config.mamba_block_size is not None\n            and self.cache_config.mamba_block_size != self.model_config.max_model_len\n        )\n        if mamba_block_size_is_set and not self.cache_config.enable_prefix_caching:\n            raise ValueError(\n                \"--mamba-block-size can only be set with --enable-prefix-caching\"\n            )\n        return self\n\n    @model_validator(mode=\"after\")\n    def validate_mamba_cached_kernel(self) -> \"VllmConfig\":\n        if not self.cache_config.use_replayssm:\n            return self\n        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that\n        # opt in (supports_replayssm) build a consistent shape on both the layer\n        # and config paths. Reject others so the mamba page size cannot desync.\n        if self.model_config is not None and not self.model_config.supports_replayssm:\n            raise ValueError(\n                \"--use-replayssm is only supported for Nemotron-H models \"\n                f\"(got architecture {self.model_config.architecture!r})\"\n            )\n        if self.cache_config.mamba_cache_mode == \"all\":\n            raise ValueError(\n                \"--use-replayssm supports prefix caching only in align mode; \"\n                \"pass --mamba-cache-mode align\"\n            )\n        if self.num_speculative_tokens > 0:\n            raise ValueError(\"--use-replayssm does not support speculative decoding\")\n        if self.mamba_config.backend != MambaBackendEnum.TRITON:\n            raise ValueError(\"--use-replayssm requires --mamba-backend triton\")\n        if (\n            self.kv_transfer_config is not None\n            and self.kv_transfer_config.is_kv_transfer_instance\n        ):\n            raise ValueError(\n                \"--use-replayssm is incompatible with KV connectors \"","sourceCodeStart":2548,"sourceCodeEnd":2584,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2548-L2584","documentation":"ReplaySSM (cached mamba kernel path, `--use-replayssm`) appends a 3-tensor ring to the mamba state, and only models that opted in (`model_config.supports_replayssm`, currently Nemotron-H architectures) build consistent state shapes. Any other architecture is rejected so the mamba page size cannot desync between layer and config paths.","triggerScenarios":"Launching with `--use-replayssm` on a model whose architecture is not Nemotron-H (e.g. Mamba2, Falcon-H1, Jamba); the error prints the offending `architecture` value.","commonSituations":"Testing a mamba speed optimization flag on whatever mamba-family model is at hand; sharing a tuned Nemotron-H launch config with other hybrid models.","solutions":["Remove `--use-replayssm` for non-Nemotron-H models.","Or switch to a Nemotron-H architecture model if the replay/cached kernel path is required.","Note the sibling constraints: replayssm also requires `--mamba-cache-mode align`, no speculative decoding, and `--mamba-backend triton`."],"exampleFix":"# before (architecture='Mamba2ForCausalLM')\nvllm serve state-spaces/mamba-2.7b --use-replayssm\n\n# after\nvllm serve state-spaces/mamba-2.7b   # flag removed\n# or\nvllm serve nvidia/Nemotron-H-8B-v1 --use-replayssm \\\n  --mamba-cache-mode align --mamba-backend triton","handlingStrategy":"validation","validationCode":"NEMOTRON_H = \"NemotronHForCausalLM\"\nif use_replayssm and model_config.architecture != NEMOTRON_H:\n    use_replayssm = False","typeGuard":"def supports_replayssm(architecture: str) -> bool:\n    return architecture == \"NemotronHForCausalLM\"","tryCatchPattern":"try:\n    LLM(model=model, use_replayssm=True, ...)\nexcept ValueError as e:\n    if \"--use-replayssm is only supported\" in str(e):\n        LLM(model=model, ...)\n    else:\n        raise","preventionTips":["Bind --use-replayssm to Nemotron-H model presets only","Remember replayssm also needs align cache mode, triton backend, no spec decode"],"tags":["mamba","replayssm","nemotron","model-support"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}