vllm-project/vllm · error · ValueError

--use-replayssm requires --mamba-backend triton

Error message

--use-replayssm requires --mamba-backend triton

What it means

Raised by VllmConfig.validate_mamba_cached_kernel when ReplaySSM (--use-replayssm) is enabled but the mamba backend is not the Triton one (mamba_config.backend != MambaBackendEnum.TRITON). The ReplaySSM replay path is only implemented against the Triton mamba kernels, so other backends (e.g. the CUDA/FLA variants) are rejected.

Source

Thrown at vllm/config/vllm.py:2578

        if not self.cache_config.use_replayssm:
            return self
        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that
        # opt in (supports_replayssm) build a consistent shape on both the layer
        # and config paths. Reject others so the mamba page size cannot desync.
        if self.model_config is not None and not self.model_config.supports_replayssm:
            raise ValueError(
                "--use-replayssm is only supported for Nemotron-H models "
                f"(got architecture {self.model_config.architecture!r})"
            )
        if self.cache_config.mamba_cache_mode == "all":
            raise ValueError(
                "--use-replayssm supports prefix caching only in align mode; "
                "pass --mamba-cache-mode align"
            )
        if self.num_speculative_tokens > 0:
            raise ValueError("--use-replayssm does not support speculative decoding")
        if self.mamba_config.backend != MambaBackendEnum.TRITON:
            raise ValueError("--use-replayssm requires --mamba-backend triton")
        if (
            self.kv_transfer_config is not None
            and self.kv_transfer_config.is_kv_transfer_instance
        ):
            raise ValueError(
                "--use-replayssm is incompatible with KV connectors "
                "(P/D disaggregation, KV cache offload)"
            )
        return self


_current_vllm_config: VllmConfig | None = None
_current_prefix: str | None = None


@contextmanager
def set_current_vllm_config(
    vllm_config: VllmConfig, check_compile=False, prefix: str | None = None

View on GitHub (pinned to c794754062)

Solutions

  1. Pass --mamba-backend triton together with --use-replayssm.
  2. Check for typos in the backend name; the value must map to MambaBackendEnum.TRITON.

Example fix

# before
vllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm --mamba-backend cuda
# after
vllm serve NVIDIA/Nemotron-H-8B-V1 --use-replayssm --mamba-backend triton
Defensive patterns

Strategy: validation

Validate before calling

if cfg.cache_config.use_replayssm and str(cfg.mamba_config.backend).lower() != "triton":
    raise SystemExit("--use-replayssm requires --mamba-backend triton")

Type guard

def replayssm_backend_ok(cfg) -> bool:
    from vllm.config.mamba import MambaBackendEnum
    return (not cfg.cache_config.use_replayssm
            or cfg.mamba_config.backend == MambaBackendEnum.TRITON)

Try / catch

try:
    cfg = VllmConfig.from_cli(args)
except ValueError as e:
    if "mamba-backend" in str(e):
        args["mamba_backend"] = "triton"; revalidate(args)
    raise

Prevention

When it happens

Trigger: Launching with --use-replayssm while --mamba-backend is set to a non-triton value, or defaults to one in the current build.

Common situations: Reusing a config tuned for the CUDA mamba backend and adding --use-replayssm on top; a platform where the default mamba backend is not Triton.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/ed345fc183049c1a. Report an issue: GitHub.