{"record":{"id":"a52a32679577d194","repo":"huggingface/transformers","slug":"invalid-max-matching-ngram-size-or-num-output-toke","errorCode":null,"errorMessage":"Invalid max_matching_ngram_size or num_output_tokens","messagePattern":"Invalid max_matching_ngram_size or num_output_tokens","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/candidate_generator.py","lineNumber":1055,"sourceCode":"\n    def __init__(\n        self,\n        eos_token_id: torch.Tensor | None = None,\n        num_output_tokens: int = 10,\n        max_matching_ngram_size: int = 2,\n        max_length: int = 20,\n        logits_processor: Optional[\"LogitsProcessorList\"] = None,\n        vocab_size: int | None = None,\n    ):\n        self.num_output_tokens = num_output_tokens\n        self.max_matching_ngram_size = max_matching_ngram_size\n        self.max_length = max_length\n        self.eos_token_id = eos_token_id\n        self.logits_processor = logits_processor\n        self.vocab_size = vocab_size\n\n        if self.max_matching_ngram_size <= 0 or self.num_output_tokens <= 0:\n            raise ValueError(\"Invalid max_matching_ngram_size or num_output_tokens\")\n\n    def get_candidates(self, input_ids: torch.LongTensor, **kwargs) -> tuple[torch.LongTensor, torch.FloatTensor]:\n        \"\"\"\n        Fetches the candidates to be tried for the current input.\n\n        Args:\n            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):\n                Indices of input sequence tokens in the vocabulary. [What are input IDs?](../glossary#input-ids)\n\n        Return:\n            `torch.LongTensor` of shape `(num_candidates, candidate_length)`: The candidate sequences to be tried.\n        \"\"\"\n        bsz, input_length = input_ids.shape\n\n        # Don't generate more than `max_length - 1` candidates since the target model generates one extra token.\n        if self.max_length == input_length + 1:\n            return input_ids, None\n","sourceCodeStart":1037,"sourceCodeEnd":1073,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/candidate_generator.py#L1037-L1073","documentation":"PromptLookupCandidateGenerator (ngram prompt lookup, used by num_assistant_tokens_schedule / prompt_lookup generation) requires both max_matching_ngram_size and num_output_tokens (candidate length) to be strictly positive. Zero or negative values make the generator unable to slice n-grams or emit candidates, so the constructor rejects them.","triggerScenarios":"model.generate(..., prompt_lookup_num_tokens=0) or negative; or constructing PromptLookupCandidateGenerator(..., max_matching_ngram_size=0, num_output_tokens=0). Also num_output_tokens computed as 0 from generation_config values.","commonSituations":"Passing 0 intending to 'disable' the feature (use None or omit instead), off-by-one configs copied from examples, or programmatically derived candidate counts that can reach 0.","solutions":["Set both values to >= 1 (typical: max_matching_ngram_size=2-4, num_output_tokens/prompt_lookup_num_tokens=10)","To disable prompt lookup, omit the argument / set prompt_lookup_num_tokens=None rather than 0","Clamp programmatically computed values: max(1, n)"],"exampleFix":"# before\nout = model.generate(inputs, prompt_lookup_num_tokens=0, do_sample=False)\n# after\nout = model.generate(inputs, prompt_lookup_num_tokens=10, do_sample=False)","handlingStrategy":"validation","validationCode":"def valid_lookup_params(max_matching_ngram_size: int, num_output_tokens: int) -> bool:\n    return max_matching_ngram_size > 0 and num_output_tokens > 0","typeGuard":"def is_positive_int(v) -> bool:\n    return isinstance(v, int) and not isinstance(v, bool) and v > 0","tryCatchPattern":null,"preventionTips":["Use prompt_lookup_num_tokens=None (or omit it) to disable prompt lookup, never 0","Clamp derived values with max(1, n) before passing them in"],"tags":["python","transformers","generation","prompt-lookup","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}