{"record":{"id":"e4968f0f01e34c2e","repo":"huggingface/transformers","slug":"setting-assistant-ensemble-weight-requires-candi","errorCode":null,"errorMessage":"Setting `assistant_ensemble_weight` requires candidate logits from the assistant model. It is not supported with prompt lookup decoding.","messagePattern":"Setting `assistant_ensemble_weight` requires candidate logits from the assistant model\\. It is not supported with prompt lookup decoding\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":1023,"sourceCode":"        assistant_tokenizer: Optional[\"PreTrainedTokenizerBase\"] = None,\n    ) -> CandidateGenerator:\n        \"\"\"\n        Returns the candidate generator to be used in `assisted_generation`\n        \"\"\"\n        different_tokenizers = all(v is not None for v in (assistant_model, target_tokenizer, assistant_tokenizer))\n\n        if generation_config.assistant_early_exit is not None:\n            candidate_generator = EarlyExitCandidateGenerator(\n                input_ids=input_ids,\n                assistant_model=self,\n                generation_config=generation_config,\n                model_kwargs=model_kwargs,\n                inputs_tensor=inputs_tensor,\n                logits_processor=logits_processor,\n            )\n        elif generation_config.prompt_lookup_num_tokens is not None:\n            if generation_config.assistant_ensemble_weight is not None:\n                raise ValueError(\n                    \"Setting `assistant_ensemble_weight` requires candidate logits from the assistant model. \"\n                    \"It is not supported with prompt lookup decoding.\"\n                )\n            candidate_generator = PromptLookupCandidateGenerator(\n                eos_token_id=generation_config._eos_token_tensor,\n                num_output_tokens=generation_config.prompt_lookup_num_tokens,\n                max_matching_ngram_size=generation_config.max_matching_ngram_size or 2,\n                max_length=generation_config.max_length,\n                logits_processor=logits_processor,\n                vocab_size=self.config.get_text_config().vocab_size,\n            )\n        elif generation_config.use_mtp:\n            candidate_generator = MTPCandidateGenerator(\n                main_model=self,\n                generation_config=generation_config,\n                logits_processor=logits_processor,\n                model_kwargs=model_kwargs,\n            )","sourceCodeStart":1005,"sourceCodeEnd":1041,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L1005-L1041","documentation":"ValueError while selecting the candidate generator for assisted generation: assistant_ensemble_weight requires logits from a real assistant model to ensemble candidate scores, but prompt_lookup_num_tokens switches to PromptLookupCandidateGenerator (n-gram copying from the prompt), which produces no assistant logits. The two options are mutually exclusive.","triggerScenarios":"model.generate(..., assistant_model=None, prompt_lookup_num_tokens=10, assistant_ensemble_weight=0.5); a generation_config that sets both keys; enabling PLD in a config that already carries an ensemble weight from a distilled-assistant setup.","commonSituations":"Copy-pasting speculative-decoding configs; frameworks toggling prompt lookup for speed while leaving assistant_ensemble_weight set; defaults in generation_config.json of an assistant-enabled checkpoint.","solutions":["Remove assistant_ensemble_weight (set it to None) when using prompt_lookup_num_tokens.","Or drop prompt_lookup_num_tokens and pass a real assistant_model if you need ensemble weighting.","Inspect model.generation_config for both fields before generate and sanitize."],"exampleFix":"# before\nout = model.generate(**inputs, prompt_lookup_num_tokens=10, assistant_ensemble_weight=0.5)\n\n# after\nout = model.generate(**inputs, prompt_lookup_num_tokens=10)","handlingStrategy":"validation","validationCode":"gc = model.generation_config\nif gc.prompt_lookup_num_tokens is not None and gc.assistant_ensemble_weight is not None:\n    gc.assistant_ensemble_weight = None  # PLD and ensemble weighting are mutually exclusive","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Sanitize generation_config before generate when enabling prompt lookup decoding.","Know the assisted-generation matrix: prompt_lookup OR assistant_model+ensemble, never both."],"tags":["generate","assisted-generation","speculative-decoding","prompt-lookup","config-conflict"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}