{"record":{"id":"bd8beb6fccb0e82b","repo":"vllm-project/vllm","slug":"sampling-distribution-replay-does-not-support-diff","errorCode":null,"errorMessage":"sampling distribution replay does not support diffusion models","messagePattern":"sampling distribution replay does not support diffusion models","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":1037,"sourceCode":"            \"(e.g. IB memory regions registered by NIXL or Mooncake). Either \"\n            \"unset expandable_segments:True or enable the cumem allocator \"\n            \"(sleep mode does this automatically and also \"\n            \"routes KV allocations through CuMemAllocator's pool, where \"\n            \"expandable_segments is automatically disabled).\"\n        )\n\n    def _verify_sampling_replay_config(self) -> None:\n        model_config = self.model_config\n        if model_config is None or not model_config.return_sampling_mask:\n            return\n        if not self.use_v2_model_runner:\n            raise ValueError(\"sampling distribution replay requires Model Runner V2\")\n        if self.speculative_config is not None:\n            raise ValueError(\n                \"sampling distribution replay does not support speculative decoding\"\n            )\n        if model_config.is_diffusion:\n            raise ValueError(\n                \"sampling distribution replay does not support diffusion models\"\n            )\n        if model_config.logits_processors:\n            raise ValueError(\n                \"sampling distribution replay does not support custom logits processors\"\n            )\n        if model_config.logprobs_mode != \"processed_logprobs\":\n            raise ValueError(\n                \"sampling distribution replay requires \"\n                \"logprobs_mode='processed_logprobs' so that returned logprobs \"\n                \"are normalized over the same nucleus as the sampling mask\"\n            )\n\n    def __post_init__(self):\n        \"\"\"Verify configs are valid & consistent with each other.\"\"\"\n\n        # To give each torch profile run a unique instance name.\n        self.instance_id = f\"{time.time_ns()}\"","sourceCodeStart":1019,"sourceCodeEnd":1055,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L1019-L1055","documentation":"The sampling-replay validator refuses diffusion models (model_config.is_diffusion truthy) when return_sampling_mask is set. Diffusion generation produces tokens through iterative denoising rather than a single next-token probability distribution, so there is no per-step categorical sampling mask to return or replay. The check is part of the same _verify_sampling_replay_config chain, evaluated after the V2 and spec-decode checks.","triggerScenarios":"Loading a diffusion LM architecture (flagged by model_config.is_diffusion) with --return-sampling-mask enabled on a V2 runner.","commonSituations":"Experimenting with sampling replay on diffusion-language models; reusing a distillation-collection config against a new diffusion checkpoint; enabling the mask globally without per-model-type gating.","solutions":["Disable return_sampling_mask for diffusion models — there is no categorical distribution to replay","Collect diffusion internals via the model's own logging/intermediate outputs instead","Route diffusion models to a separate deployment profile without the flag"],"exampleFix":"# before\nvllm serve diffusion-lm --return-sampling-mask\n# after\nvllm serve diffusion-lm","handlingStrategy":"validation","validationCode":"if model_cfg.get(\"return_sampling_mask\"):\n    assert not getattr(model_config, \"is_diffusion\", False), \\\n        \"sampling replay unsupported for diffusion models\"","typeGuard":"def replay_diffusion_ok(return_mask: bool, is_diffusion: bool) -> bool:\n    return not return_mask or not is_diffusion","tryCatchPattern":null,"preventionTips":["Tag diffusion checkpoints in your model registry and auto-strip mask/replay flags"],"tags":["sampling","diffusion","config","incompatibility"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}