{"record":{"id":"2844f4a584889da7","repo":"vllm-project/vllm","slug":"use-replayssm-does-not-support-speculative-decod","errorCode":null,"errorMessage":"--use-replayssm does not support speculative decoding","messagePattern":"--use-replayssm does not support speculative decoding","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2576,"sourceCode":"    @model_validator(mode=\"after\")\n    def validate_mamba_cached_kernel(self) -> \"VllmConfig\":\n        if not self.cache_config.use_replayssm:\n            return self\n        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that\n        # opt in (supports_replayssm) build a consistent shape on both the layer\n        # and config paths. Reject others so the mamba page size cannot desync.\n        if self.model_config is not None and not self.model_config.supports_replayssm:\n            raise ValueError(\n                \"--use-replayssm is only supported for Nemotron-H models \"\n                f\"(got architecture {self.model_config.architecture!r})\"\n            )\n        if self.cache_config.mamba_cache_mode == \"all\":\n            raise ValueError(\n                \"--use-replayssm supports prefix caching only in align mode; \"\n                \"pass --mamba-cache-mode align\"\n            )\n        if self.num_speculative_tokens > 0:\n            raise ValueError(\"--use-replayssm does not support speculative decoding\")\n        if self.mamba_config.backend != MambaBackendEnum.TRITON:\n            raise ValueError(\"--use-replayssm requires --mamba-backend triton\")\n        if (\n            self.kv_transfer_config is not None\n            and self.kv_transfer_config.is_kv_transfer_instance\n        ):\n            raise ValueError(\n                \"--use-replayssm is incompatible with KV connectors \"\n                \"(P/D disaggregation, KV cache offload)\"\n            )\n        return self\n\n\n_current_vllm_config: VllmConfig | None = None\n_current_prefix: str | None = None\n\n\n@contextmanager","sourceCodeStart":2558,"sourceCodeEnd":2594,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2558-L2594","documentation":"Raised by VllmConfig.validate_mamba_cached_kernel when ReplaySSM (--use-replayssm) is combined with speculative decoding (num_speculative_tokens > 0). The replayed SSM state ring cannot be kept consistent when draft tokens are replayed, so vLLM rejects the combination during config validation.","triggerScenarios":"Launching with --use-replayssm plus any speculative-decoding setup that makes num_speculative_tokens > 0 (e.g. a spec-decode config such as ngram, eagle, or medusa with num_speculative_tokens set).","commonSituations":"A serving config that already had speculative decoding enabled is extended with --use-replayssm; using a shared benchmarking script that always turns on spec decode.","solutions":["Disable speculative decoding: remove/zero the spec-decode method and num_speculative_tokens from the launch config.","If spec decode is required, drop --use-replayssm."],"exampleFix":"# before\nvllm serve model --use-replayssm \\\n  --speculative-config '{\"method\":\"ngram\",\"num_speculative_tokens\":3}'\n# after\nvllm serve model --use-replayssm","handlingStrategy":"validation","validationCode":"if cfg.num_speculative_tokens > 0 and cfg.cache_config.use_replayssm:\n    raise SystemExit(\"choose either --use-replayssm or speculative decoding, not both\")","typeGuard":"def spec_decode_compatible(cfg) -> bool:\n    return not cfg.cache_config.use_replayssm or cfg.num_speculative_tokens == 0","tryCatchPattern":"try:\n    engine = LLM(**args)\nexcept ValueError as e:\n    if \"speculative\" in str(e) and \"replayssm\" in str(e):\n        args.pop(\"speculative_config\"); retry_launch_without_spec_decode()\n    raise","preventionTips":["Do not mix spec-decode and replayssm flags in shared launcher scripts.","Add a config linter that rejects the known-incompatible flag pairs before submission."],"tags":["vllm","config","replayssm","speculative-decoding","mamba"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}