vllm-project/vllm · error · ValueError

sampling distribution replay does not support diffusion mode

Error message

sampling distribution replay does not support diffusion models

What it means

The sampling-replay validator refuses diffusion models (model_config.is_diffusion truthy) when return_sampling_mask is set. Diffusion generation produces tokens through iterative denoising rather than a single next-token probability distribution, so there is no per-step categorical sampling mask to return or replay. The check is part of the same _verify_sampling_replay_config chain, evaluated after the V2 and spec-decode checks.

Source

Thrown at vllm/config/vllm.py:1037

            "(e.g. IB memory regions registered by NIXL or Mooncake). Either "
            "unset expandable_segments:True or enable the cumem allocator "
            "(sleep mode does this automatically and also "
            "routes KV allocations through CuMemAllocator's pool, where "
            "expandable_segments is automatically disabled)."
        )

    def _verify_sampling_replay_config(self) -> None:
        model_config = self.model_config
        if model_config is None or not model_config.return_sampling_mask:
            return
        if not self.use_v2_model_runner:
            raise ValueError("sampling distribution replay requires Model Runner V2")
        if self.speculative_config is not None:
            raise ValueError(
                "sampling distribution replay does not support speculative decoding"
            )
        if model_config.is_diffusion:
            raise ValueError(
                "sampling distribution replay does not support diffusion models"
            )
        if model_config.logits_processors:
            raise ValueError(
                "sampling distribution replay does not support custom logits processors"
            )
        if model_config.logprobs_mode != "processed_logprobs":
            raise ValueError(
                "sampling distribution replay requires "
                "logprobs_mode='processed_logprobs' so that returned logprobs "
                "are normalized over the same nucleus as the sampling mask"
            )

    def __post_init__(self):
        """Verify configs are valid & consistent with each other."""

        # To give each torch profile run a unique instance name.
        self.instance_id = f"{time.time_ns()}"

View on GitHub (pinned to c794754062)

Solutions

1. Disable return_sampling_mask for diffusion models — there is no categorical distribution to replay
2. Collect diffusion internals via the model's own logging/intermediate outputs instead
3. Route diffusion models to a separate deployment profile without the flag

Example fix

# before
vllm serve diffusion-lm --return-sampling-mask
# after
vllm serve diffusion-lm

if model_cfg.get("return_sampling_mask"):
    assert not getattr(model_config, "is_diffusion", False), \
        "sampling replay unsupported for diffusion models"

def replay_diffusion_ok(return_mask: bool, is_diffusion: bool) -> bool:
    return not return_mask or not is_diffusion

When it happens

Trigger: Loading a diffusion LM architecture (flagged by model_config.is_diffusion) with --return-sampling-mask enabled on a V2 runner.

Common situations: Experimenting with sampling replay on diffusion-language models; reusing a distillation-collection config against a new diffusion checkpoint; enabling the mask globally without per-model-type gating.

Related errors

• sampling distribution replay does not support speculative de
• sampling distribution replay does not support custom logits
• use_heterogeneous_vocab currently only supports greedy draft
• sampling distribution replay requires Model Runner V2
• sampling distribution replay requires logprobs_mode='process

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/bd8beb6fccb0e82b. Report an issue: GitHub.