vllm-project/vllm · error · ValueError

sampling distribution replay requires logprobs_mode='process

Error message

sampling distribution replay requires logprobs_mode='processed_logprobs' so that returned logprobs are normalized over the same nucleus as the sampling mask

What it means

The final check of _verify_sampling_replay_config requires logprobs_mode='processed_logprobs' when return_sampling_mask is enabled. The replay feature promises that returned logprobs are normalized over exactly the same nucleus (top-p / top-k truncation) as the sampling mask; only the 'processed_logprobs' mode applies that truncation before returning. Other logprobs modes return raw or differently-normalized distributions, which would make mask and logprobs inconsistent.

Source

Thrown at vllm/config/vllm.py:1045

        model_config = self.model_config
        if model_config is None or not model_config.return_sampling_mask:
            return
        if not self.use_v2_model_runner:
            raise ValueError("sampling distribution replay requires Model Runner V2")
        if self.speculative_config is not None:
            raise ValueError(
                "sampling distribution replay does not support speculative decoding"
            )
        if model_config.is_diffusion:
            raise ValueError(
                "sampling distribution replay does not support diffusion models"
            )
        if model_config.logits_processors:
            raise ValueError(
                "sampling distribution replay does not support custom logits processors"
            )
        if model_config.logprobs_mode != "processed_logprobs":
            raise ValueError(
                "sampling distribution replay requires "
                "logprobs_mode='processed_logprobs' so that returned logprobs "
                "are normalized over the same nucleus as the sampling mask"
            )

    def __post_init__(self):
        """Verify configs are valid & consistent with each other."""

        # To give each torch profile run a unique instance name.
        self.instance_id = f"{time.time_ns()}"

        if self.performance_mode != "balanced":
            logger.info_once("Performance mode set to '%s'.", self.performance_mode)

        self.try_verify_and_update_config()

        if self.model_config is not None:
            self.model_config.verify_with_parallel_config(self.parallel_config)

View on GitHub (pinned to c794754062)

Solutions

1. Set logprobs_mode='processed_logprobs' (e.g. --logprobs-mode processed_logprobs) alongside --return-sampling-mask
2. If you need raw logprobs, disable return_sampling_mask on that instance

Example fix

# before
vllm serve model --return-sampling-mask --logprobs-mode raw_logprobs
# after
vllm serve model --return-sampling-mask --logprobs-mode processed_logprobs

if model_cfg.get("return_sampling_mask"):
    assert model_cfg.get("logprobs_mode", "processed_logprobs") == "processed_logprobs", \
        "mask replay needs processed_logprobs"

def replay_logprobs_ok(return_mask: bool, mode: str) -> bool:
    return not return_mask or mode == "processed_logprobs"

When it happens

Trigger: Enabling --return-sampling-mask while logprobs_mode is left at its default (e.g. 'raw_logprobs') or explicitly set to anything other than 'processed_logprobs'.

Common situations: Setting logprobs_mode for API compatibility (raw logprobs) and then turning on mask return; partial configs where one flag comes from a template and the other from CLI; assuming any logprobs mode pairs with the mask.

Related errors

• use_heterogeneous_vocab currently only supports greedy draft
• sampling distribution replay requires Model Runner V2
• max_logprobs must be non-negative or -1, got {}
• max_logprobs must be non-negative or -1
• {parameter} must be non-negative or -1, got {value}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/bd235de2e8d2c629. Report an issue: GitHub.