vllm-project/vllm · error · ValueError

--use-replayssm is incompatible with KV connectors (P/D disa

Error message

--use-replayssm is incompatible with KV connectors (P/D disaggregation, KV cache offload)

What it means

Raised by VllmConfig.validate_mamba_cached_kernel when ReplaySSM (--use-replayssm) is combined with an active KV connector (kv_transfer_config is not None and is_kv_transfer_instance is true), i.e. P/D disaggregation or KV cache offload. ReplaySSM's mamba-state ring has no representation in the KV-transfer protocol, so the combination is rejected.

Source

Thrown at vllm/config/vllm.py:2583

        if self.model_config is not None and not self.model_config.supports_replayssm:
            raise ValueError(
                "--use-replayssm is only supported for Nemotron-H models "
                f"(got architecture {self.model_config.architecture!r})"
            )
        if self.cache_config.mamba_cache_mode == "all":
            raise ValueError(
                "--use-replayssm supports prefix caching only in align mode; "
                "pass --mamba-cache-mode align"
            )
        if self.num_speculative_tokens > 0:
            raise ValueError("--use-replayssm does not support speculative decoding")
        if self.mamba_config.backend != MambaBackendEnum.TRITON:
            raise ValueError("--use-replayssm requires --mamba-backend triton")
        if (
            self.kv_transfer_config is not None
            and self.kv_transfer_config.is_kv_transfer_instance
        ):
            raise ValueError(
                "--use-replayssm is incompatible with KV connectors "
                "(P/D disaggregation, KV cache offload)"
            )
        return self


_current_vllm_config: VllmConfig | None = None
_current_prefix: str | None = None


@contextmanager
def set_current_vllm_config(
    vllm_config: VllmConfig, check_compile=False, prefix: str | None = None
):
    """
    Temporarily set the current vLLM config.
    Used during model initialization.
    We save the current vLLM config in a global variable,

View on GitHub (pinned to c794754062)

Solutions

Remove or disable the KV transfer config (--kv-transfer-config and any offload connector) when using --use-replayssm.
If P/D disaggregation or offload is mandatory, run without --use-replayssm.
Inspect the effective cache_config/model_config at startup to confirm no plugin silently registered a connector.

Example fix

# before
vllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm \
  --kv-transfer-config '{"kv_connector":"SharedStorageConnector",...}'
# after
vllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm

Defensive patterns

Strategy: validation

Validate before calling

kv = getattr(cfg, "kv_transfer_config", None)
if cfg.cache_config.use_replayssm and kv is not None and kv.is_kv_transfer_instance:
    raise SystemExit("--use-replayssm is incompatible with KV connectors; remove --kv-transfer-config")

Type guard

def kv_connector_active(cfg) -> bool:
    kv = getattr(cfg, "kv_transfer_config", None)
    return kv is not None and kv.is_kv_transfer_instance

Try / catch

try:
    engine_args = EngineArgs(**cli)
except ValueError as e:
    if "KV connectors" in str(e):
        cli.pop("kv_transfer_config", None); rebuild_engine_args(cli)
    raise

Prevention

Audit plugins/env (e.g. offload connectors) that can inject kv_transfer_config implicitly.
Maintain separate profiles for disaggregated stacks vs replayssm stacks; never merge them.

When it happens

Trigger: Launching with --use-replayssm plus --kv-transfer-config (or a kv-cache-offload connector) that resolves to a KV transfer instance.

Common situations: Adding ReplaySSM to a disaggregated serving stack (prefill/decode separation) or one using LMCache/offload connectors; the KV connector is often injected by a plugin or by environment defaults rather than an explicit flag.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/73ef6c482e581931. Report an issue: GitHub.