{"record":{"id":"73ef6c482e581931","repo":"vllm-project/vllm","slug":"use-replayssm-is-incompatible-with-kv-connectors","errorCode":null,"errorMessage":"--use-replayssm is incompatible with KV connectors (P/D disaggregation, KV cache offload)","messagePattern":"--use-replayssm is incompatible with KV connectors \\(P/D disaggregation, KV cache offload\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":2583,"sourceCode":"        if self.model_config is not None and not self.model_config.supports_replayssm:\n            raise ValueError(\n                \"--use-replayssm is only supported for Nemotron-H models \"\n                f\"(got architecture {self.model_config.architecture!r})\"\n            )\n        if self.cache_config.mamba_cache_mode == \"all\":\n            raise ValueError(\n                \"--use-replayssm supports prefix caching only in align mode; \"\n                \"pass --mamba-cache-mode align\"\n            )\n        if self.num_speculative_tokens > 0:\n            raise ValueError(\"--use-replayssm does not support speculative decoding\")\n        if self.mamba_config.backend != MambaBackendEnum.TRITON:\n            raise ValueError(\"--use-replayssm requires --mamba-backend triton\")\n        if (\n            self.kv_transfer_config is not None\n            and self.kv_transfer_config.is_kv_transfer_instance\n        ):\n            raise ValueError(\n                \"--use-replayssm is incompatible with KV connectors \"\n                \"(P/D disaggregation, KV cache offload)\"\n            )\n        return self\n\n\n_current_vllm_config: VllmConfig | None = None\n_current_prefix: str | None = None\n\n\n@contextmanager\ndef set_current_vllm_config(\n    vllm_config: VllmConfig, check_compile=False, prefix: str | None = None\n):\n    \"\"\"\n    Temporarily set the current vLLM config.\n    Used during model initialization.\n    We save the current vLLM config in a global variable,","sourceCodeStart":2565,"sourceCodeEnd":2601,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L2565-L2601","documentation":"Raised by VllmConfig.validate_mamba_cached_kernel when ReplaySSM (--use-replayssm) is combined with an active KV connector (kv_transfer_config is not None and is_kv_transfer_instance is true), i.e. P/D disaggregation or KV cache offload. ReplaySSM's mamba-state ring has no representation in the KV-transfer protocol, so the combination is rejected.","triggerScenarios":"Launching with --use-replayssm plus --kv-transfer-config (or a kv-cache-offload connector) that resolves to a KV transfer instance.","commonSituations":"Adding ReplaySSM to a disaggregated serving stack (prefill/decode separation) or one using LMCache/offload connectors; the KV connector is often injected by a plugin or by environment defaults rather than an explicit flag.","solutions":["Remove or disable the KV transfer config (--kv-transfer-config and any offload connector) when using --use-replayssm.","If P/D disaggregation or offload is mandatory, run without --use-replayssm.","Inspect the effective cache_config/model_config at startup to confirm no plugin silently registered a connector."],"exampleFix":"# before\nvllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm \\\n  --kv-transfer-config '{\"kv_connector\":\"SharedStorageConnector\",...}'\n# after\nvllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm","handlingStrategy":"validation","validationCode":"kv = getattr(cfg, \"kv_transfer_config\", None)\nif cfg.cache_config.use_replayssm and kv is not None and kv.is_kv_transfer_instance:\n    raise SystemExit(\"--use-replayssm is incompatible with KV connectors; remove --kv-transfer-config\")","typeGuard":"def kv_connector_active(cfg) -> bool:\n    kv = getattr(cfg, \"kv_transfer_config\", None)\n    return kv is not None and kv.is_kv_transfer_instance","tryCatchPattern":"try:\n    engine_args = EngineArgs(**cli)\nexcept ValueError as e:\n    if \"KV connectors\" in str(e):\n        cli.pop(\"kv_transfer_config\", None); rebuild_engine_args(cli)\n    raise","preventionTips":["Audit plugins/env (e.g. offload connectors) that can inject kv_transfer_config implicitly.","Maintain separate profiles for disaggregated stacks vs replayssm stacks; never merge them."],"tags":["vllm","config","replayssm","kv-transfer","disaggregation","mamba"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}