{"record":{"id":"5e47c6c48aadc4e3","repo":"vllm-project/vllm","slug":"sampling-distribution-replay-does-not-support-cust","errorCode":null,"errorMessage":"sampling distribution replay does not support custom logits processors","messagePattern":"sampling distribution replay does not support custom logits processors","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":1041,"sourceCode":"            \"expandable_segments is automatically disabled).\"\n        )\n\n    def _verify_sampling_replay_config(self) -> None:\n        model_config = self.model_config\n        if model_config is None or not model_config.return_sampling_mask:\n            return\n        if not self.use_v2_model_runner:\n            raise ValueError(\"sampling distribution replay requires Model Runner V2\")\n        if self.speculative_config is not None:\n            raise ValueError(\n                \"sampling distribution replay does not support speculative decoding\"\n            )\n        if model_config.is_diffusion:\n            raise ValueError(\n                \"sampling distribution replay does not support diffusion models\"\n            )\n        if model_config.logits_processors:\n            raise ValueError(\n                \"sampling distribution replay does not support custom logits processors\"\n            )\n        if model_config.logprobs_mode != \"processed_logprobs\":\n            raise ValueError(\n                \"sampling distribution replay requires \"\n                \"logprobs_mode='processed_logprobs' so that returned logprobs \"\n                \"are normalized over the same nucleus as the sampling mask\"\n            )\n\n    def __post_init__(self):\n        \"\"\"Verify configs are valid & consistent with each other.\"\"\"\n\n        # To give each torch profile run a unique instance name.\n        self.instance_id = f\"{time.time_ns()}\"\n\n        if self.performance_mode != \"balanced\":\n            logger.info_once(\"Performance mode set to '%s'.\", self.performance_mode)\n","sourceCodeStart":1023,"sourceCodeEnd":1059,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L1023-L1059","documentation":"Sampling-distribution replay is incompatible with custom logits processors (model_config.logits_processors non-empty when return_sampling_mask is set). Custom processors mutate the logits distribution after the model forward, so the returned mask would reflect the processed distribution while consumers assume the model's native nucleus — replaying it elsewhere would be wrong. The validator fails fast at config time.","triggerScenarios":"Registering --logits-processors my_module.processors (or passing logits_processors in model config) together with --return-sampling-mask on Model Runner V2.","commonSituations":"Distillation pipelines that add temperature/logit-bias processors on the teacher; porting an existing server (which already used logits processors for censorship/bias) to also emit sampling masks; third-party plugins that inject processors via config silently.","solutions":["Remove the custom logits processors for the instance that returns sampling masks","Apply the desired transformation on the client side using the returned logprobs/mask instead of server-side processors","Run a separate no-processor deployment for mask collection"],"exampleFix":"# before\nvllm serve model --return-sampling-mask --logits-processors myplugin.bias\n# after\nvllm serve model --return-sampling-mask  # apply bias client-side from returned logprobs","handlingStrategy":"validation","validationCode":"if model_cfg.get(\"return_sampling_mask\"):\n    assert not model_config.logits_processors, \\\n        \"sampling replay incompatible with custom logits processors\"","typeGuard":"def replay_processors_ok(return_mask: bool, processors) -> bool:\n    return not return_mask or not processors","tryCatchPattern":null,"preventionTips":["Audit configs for plugin-injected logits processors before enabling mask return","Move distribution post-processing client-side when collecting masks"],"tags":["sampling","logits-processors","config","incompatibility"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}