vllm-project/vllm · error · ValueError

--use-replayssm is only supported for Nemotron-H models (got

Error message

--use-replayssm is only supported for Nemotron-H models (got architecture {self.model_config.architecture!r})

What it means

ReplaySSM (cached mamba kernel path, `--use-replayssm`) appends a 3-tensor ring to the mamba state, and only models that opted in (`model_config.supports_replayssm`, currently Nemotron-H architectures) build consistent state shapes. Any other architecture is rejected so the mamba page size cannot desync between layer and config paths.

Source

Thrown at vllm/config/vllm.py:2566

        mamba_block_size_is_set = (
            self.cache_config.mamba_block_size is not None
            and self.cache_config.mamba_block_size != self.model_config.max_model_len
        )
        if mamba_block_size_is_set and not self.cache_config.enable_prefix_caching:
            raise ValueError(
                "--mamba-block-size can only be set with --enable-prefix-caching"
            )
        return self

    @model_validator(mode="after")
    def validate_mamba_cached_kernel(self) -> "VllmConfig":
        if not self.cache_config.use_replayssm:
            return self
        # ReplaySSM adds a 3-tensor ring to the mamba state; only models that
        # opt in (supports_replayssm) build a consistent shape on both the layer
        # and config paths. Reject others so the mamba page size cannot desync.
        if self.model_config is not None and not self.model_config.supports_replayssm:
            raise ValueError(
                "--use-replayssm is only supported for Nemotron-H models "
                f"(got architecture {self.model_config.architecture!r})"
            )
        if self.cache_config.mamba_cache_mode == "all":
            raise ValueError(
                "--use-replayssm supports prefix caching only in align mode; "
                "pass --mamba-cache-mode align"
            )
        if self.num_speculative_tokens > 0:
            raise ValueError("--use-replayssm does not support speculative decoding")
        if self.mamba_config.backend != MambaBackendEnum.TRITON:
            raise ValueError("--use-replayssm requires --mamba-backend triton")
        if (
            self.kv_transfer_config is not None
            and self.kv_transfer_config.is_kv_transfer_instance
        ):
            raise ValueError(
                "--use-replayssm is incompatible with KV connectors "

View on GitHub (pinned to c794754062)

Solutions

  1. Remove `--use-replayssm` for non-Nemotron-H models.
  2. Or switch to a Nemotron-H architecture model if the replay/cached kernel path is required.
  3. Note the sibling constraints: replayssm also requires `--mamba-cache-mode align`, no speculative decoding, and `--mamba-backend triton`.

Example fix

# before (architecture='Mamba2ForCausalLM')
vllm serve state-spaces/mamba-2.7b --use-replayssm

# after
vllm serve state-spaces/mamba-2.7b   # flag removed
# or
vllm serve nvidia/Nemotron-H-8B-v1 --use-replayssm \
  --mamba-cache-mode align --mamba-backend triton
Defensive patterns

Strategy: validation

Validate before calling

NEMOTRON_H = "NemotronHForCausalLM"
if use_replayssm and model_config.architecture != NEMOTRON_H:
    use_replayssm = False

Type guard

def supports_replayssm(architecture: str) -> bool:
    return architecture == "NemotronHForCausalLM"

Try / catch

try:
    LLM(model=model, use_replayssm=True, ...)
except ValueError as e:
    if "--use-replayssm is only supported" in str(e):
        LLM(model=model, ...)
    else:
        raise

Prevention

When it happens

Trigger: Launching with `--use-replayssm` on a model whose architecture is not Nemotron-H (e.g. Mamba2, Falcon-H1, Jamba); the error prints the offending `architecture` value.

Common situations: Testing a mamba speed optimization flag on whatever mamba-family model is at hand; sharing a tuned Nemotron-H launch config with other hybrid models.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/1b37c46a30753abd. Report an issue: GitHub.