vllm-project/vllm · error · ValueError

sampling distribution replay does not support custom logits

Error message

sampling distribution replay does not support custom logits processors

What it means

Sampling-distribution replay is incompatible with custom logits processors (model_config.logits_processors non-empty when return_sampling_mask is set). Custom processors mutate the logits distribution after the model forward, so the returned mask would reflect the processed distribution while consumers assume the model's native nucleus — replaying it elsewhere would be wrong. The validator fails fast at config time.

Source

Thrown at vllm/config/vllm.py:1041

            "expandable_segments is automatically disabled)."
        )

    def _verify_sampling_replay_config(self) -> None:
        model_config = self.model_config
        if model_config is None or not model_config.return_sampling_mask:
            return
        if not self.use_v2_model_runner:
            raise ValueError("sampling distribution replay requires Model Runner V2")
        if self.speculative_config is not None:
            raise ValueError(
                "sampling distribution replay does not support speculative decoding"
            )
        if model_config.is_diffusion:
            raise ValueError(
                "sampling distribution replay does not support diffusion models"
            )
        if model_config.logits_processors:
            raise ValueError(
                "sampling distribution replay does not support custom logits processors"
            )
        if model_config.logprobs_mode != "processed_logprobs":
            raise ValueError(
                "sampling distribution replay requires "
                "logprobs_mode='processed_logprobs' so that returned logprobs "
                "are normalized over the same nucleus as the sampling mask"
            )

    def __post_init__(self):
        """Verify configs are valid & consistent with each other."""

        # To give each torch profile run a unique instance name.
        self.instance_id = f"{time.time_ns()}"

        if self.performance_mode != "balanced":
            logger.info_once("Performance mode set to '%s'.", self.performance_mode)

View on GitHub (pinned to c794754062)

Solutions

1. Remove the custom logits processors for the instance that returns sampling masks
2. Apply the desired transformation on the client side using the returned logprobs/mask instead of server-side processors
3. Run a separate no-processor deployment for mask collection

Example fix

# before
vllm serve model --return-sampling-mask --logits-processors myplugin.bias
# after
vllm serve model --return-sampling-mask  # apply bias client-side from returned logprobs

if model_cfg.get("return_sampling_mask"):
    assert not model_config.logits_processors, \
        "sampling replay incompatible with custom logits processors"

def replay_processors_ok(return_mask: bool, processors) -> bool:
    return not return_mask or not processors

When it happens

Trigger: Registering --logits-processors my_module.processors (or passing logits_processors in model config) together with --return-sampling-mask on Model Runner V2.

Common situations: Distillation pipelines that add temperature/logit-bias processors on the teacher; porting an existing server (which already used logits processors for censorship/bias) to also emit sampling masks; third-party plugins that inject processors via config silently.

Related errors

• sampling distribution replay does not support speculative de
• sampling distribution replay does not support diffusion mode
• use_heterogeneous_vocab currently only supports greedy draft
• sampling distribution replay requires Model Runner V2
• sampling distribution replay requires logprobs_mode='process

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/5e47c6c48aadc4e3. Report an issue: GitHub.