vllm-project/vllm · error · ValueError

--use-replayssm supports prefix caching only in align mode;

Error message

--use-replayssm supports prefix caching only in align mode; pass --mamba-cache-mode align

What it means

Raised by VllmConfig.validate_mamba_cached_kernel when the ReplaySSM feature (--use-replayssm) is enabled together with mamba_cache_mode == "all". ReplaySSM adds a 3-tensor ring to the mamba state, and prefix caching of that state is only coherent when the cache is aligned, so vLLM rejects the 'all' cache mode at config validation time.

Source

Thrown at vllm/config/vllm.py:2571

            raise ValueError(
                "--mamba-block-size can only be set with --enable-prefix-caching"
            )
        return self

    @model_validator(mode="after")
    def validate_mamba_cached_kernel(self) -> "VllmConfig":
        if not self.cache_config.use_replayssm:
            return self
        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that
        # opt in (supports_replayssm) build a consistent shape on both the layer
        # and config paths. Reject others so the mamba page size cannot desync.
        if self.model_config is not None and not self.model_config.supports_replayssm:
            raise ValueError(
                "--use-replayssm is only supported for Nemotron-H models "
                f"(got architecture {self.model_config.architecture!r})"
            )
        if self.cache_config.mamba_cache_mode == "all":
            raise ValueError(
                "--use-replayssm supports prefix caching only in align mode; "
                "pass --mamba-cache-mode align"
            )
        if self.num_speculative_tokens > 0:
            raise ValueError("--use-replayssm does not support speculative decoding")
        if self.mamba_config.backend != MambaBackendEnum.TRITON:
            raise ValueError("--use-replayssm requires --mamba-backend triton")
        if (
            self.kv_transfer_config is not None
            and self.kv_transfer_config.is_kv_transfer_instance
        ):
            raise ValueError(
                "--use-replayssm is incompatible with KV connectors "
                "(P/D disaggregation, KV cache offload)"
            )
        return self

View on GitHub (pinned to c794754062)

Solutions

Set --mamba-cache-mode align (or remove the flag if align is the default) and restart the server.
Verify the rest of the ReplaySSM constraints at the same time: Nemotron-H architecture only, no speculative decoding, --mamba-backend triton, and no KV transfer connector.

Example fix

# before
vllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm --mamba-cache-mode all
# after
vllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm --mamba-cache-mode align

Defensive patterns

Strategy: validation

Validate before calling

from vllm.config import CacheConfigMode  # illustrative
cfg = build_config(args)
if cfg.cache_config.use_replayssm and cfg.cache_config.mamba_cache_mode == "all":
    raise SystemExit("replayssm requires --mamba-cache-mode align")

Type guard

def replayssm_config_valid(cfg) -> bool:
    return not cfg.cache_config.use_replayssm or (
        cfg.cache_config.mamba_cache_mode == "align"
        and cfg.num_speculative_tokens == 0
    )

Try / catch

try:
    config = VllmConfig.from_cli(args)
except ValueError as e:
    if "replayssm" in str(e):
        log_config_and_abort(e)  # surface the conflicting flags, do not retry
    raise

Prevention

Validate the full replayssm flag set (model, cache mode, spec decode, backend, kv-transfer) in one preflight check before launching.
Keep replayssm launch profiles in a dedicated config file instead of composing flags ad hoc.

When it happens

Trigger: Starting vLLM with --use-replayssm while --mamba-cache-mode is set to (or defaults to) "all"; the check runs whenever cache_config.use_replayssm is true and cache_config.mamba_cache_mode == "all".

Common situations: Enabling ReplaySSM on a Nemotron-H model but copying mamba cache flags from a non-ReplaySSM config; using a launcher script that always passes --mamba-cache-mode all.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/fa5406aa2fe458f3. Report an issue: GitHub.