{"record":{"id":"bd235de2e8d2c629","repo":"vllm-project/vllm","slug":"sampling-distribution-replay-requires-logprobs-mod","errorCode":null,"errorMessage":"sampling distribution replay requires logprobs_mode='processed_logprobs' so that returned logprobs are normalized over the same nucleus as the sampling mask","messagePattern":"sampling distribution replay requires logprobs_mode='processed_logprobs' so that returned logprobs are normalized over the same nucleus as the sampling mask","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":1045,"sourceCode":"        model_config = self.model_config\n        if model_config is None or not model_config.return_sampling_mask:\n            return\n        if not self.use_v2_model_runner:\n            raise ValueError(\"sampling distribution replay requires Model Runner V2\")\n        if self.speculative_config is not None:\n            raise ValueError(\n                \"sampling distribution replay does not support speculative decoding\"\n            )\n        if model_config.is_diffusion:\n            raise ValueError(\n                \"sampling distribution replay does not support diffusion models\"\n            )\n        if model_config.logits_processors:\n            raise ValueError(\n                \"sampling distribution replay does not support custom logits processors\"\n            )\n        if model_config.logprobs_mode != \"processed_logprobs\":\n            raise ValueError(\n                \"sampling distribution replay requires \"\n                \"logprobs_mode='processed_logprobs' so that returned logprobs \"\n                \"are normalized over the same nucleus as the sampling mask\"\n            )\n\n    def __post_init__(self):\n        \"\"\"Verify configs are valid & consistent with each other.\"\"\"\n\n        # To give each torch profile run a unique instance name.\n        self.instance_id = f\"{time.time_ns()}\"\n\n        if self.performance_mode != \"balanced\":\n            logger.info_once(\"Performance mode set to '%s'.\", self.performance_mode)\n\n        self.try_verify_and_update_config()\n\n        if self.model_config is not None:\n            self.model_config.verify_with_parallel_config(self.parallel_config)","sourceCodeStart":1027,"sourceCodeEnd":1063,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L1027-L1063","documentation":"The final check of _verify_sampling_replay_config requires logprobs_mode='processed_logprobs' when return_sampling_mask is enabled. The replay feature promises that returned logprobs are normalized over exactly the same nucleus (top-p / top-k truncation) as the sampling mask; only the 'processed_logprobs' mode applies that truncation before returning. Other logprobs modes return raw or differently-normalized distributions, which would make mask and logprobs inconsistent.","triggerScenarios":"Enabling --return-sampling-mask while logprobs_mode is left at its default (e.g. 'raw_logprobs') or explicitly set to anything other than 'processed_logprobs'.","commonSituations":"Setting logprobs_mode for API compatibility (raw logprobs) and then turning on mask return; partial configs where one flag comes from a template and the other from CLI; assuming any logprobs mode pairs with the mask.","solutions":["Set logprobs_mode='processed_logprobs' (e.g. --logprobs-mode processed_logprobs) alongside --return-sampling-mask","If you need raw logprobs, disable return_sampling_mask on that instance"],"exampleFix":"# before\nvllm serve model --return-sampling-mask --logprobs-mode raw_logprobs\n# after\nvllm serve model --return-sampling-mask --logprobs-mode processed_logprobs","handlingStrategy":"validation","validationCode":"if model_cfg.get(\"return_sampling_mask\"):\n    assert model_cfg.get(\"logprobs_mode\", \"processed_logprobs\") == \"processed_logprobs\", \\\n        \"mask replay needs processed_logprobs\"","typeGuard":"def replay_logprobs_ok(return_mask: bool, mode: str) -> bool:\n    return not return_mask or mode == \"processed_logprobs\"","tryCatchPattern":null,"preventionTips":["Set --logprobs-mode processed_logprobs in the same template as --return-sampling-mask","Reject config combos that set the mask without the matching logprobs mode in CI lint"],"tags":["sampling","logprobs","config","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}