huggingface/transformers · error · ValueError

`ngram_size` has to be a strictly positive integer, but is {

Error message

`ngram_size` has to be a strictly positive integer, but is {ngram_size}

What it means

Thrown by NoRepeatNGramLogitsProcessor.__init__ when ngram_size is not a Python int or is <= 0. The processor bans any token that would complete an n-gram of this size already seen in the sequence, so the size must be a strictly positive integer (1 degenerates to banning every seen token and is rarely wanted).

Source

Thrown at src/transformers/generation/logits_process.py:1117

    >>> model = AutoModelForCausalLM.from_pretrained("distilbert/distilgpt2")
    >>> tokenizer = AutoTokenizer.from_pretrained("distilbert/distilgpt2")
    >>> inputs = tokenizer(["Today I"], return_tensors="pt")

    >>> output = model.generate(**inputs)
    >>> print(tokenizer.decode(output[0], skip_special_tokens=True))
    Today I'm not sure if I'm going to be able to do it.

    >>> # Now let's add ngram size using `no_repeat_ngram_size`. This stops the repetitions ("I'm") in the output.
    >>> output = model.generate(**inputs, no_repeat_ngram_size=2)
    >>> print(tokenizer.decode(output[0], skip_special_tokens=True))
    Today I'm not sure if I can get a better understanding of the nature of this issue
    ```
    """

    def __init__(self, ngram_size: int):
        if not isinstance(ngram_size, int) or ngram_size <= 0:
            raise ValueError(f"`ngram_size` has to be a strictly positive integer, but is {ngram_size}")
        self.ngram_size = ngram_size

    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        cur_len = input_ids.shape[-1]
        # No complete ngram yet, so nothing to ban
        if cur_len < self.ngram_size:
            return scores

        # An ngram can only be completed by the next token if it starts with the current suffix, so we match that one
        # prefix against every window instead of building all ngrams. A matching window bans its own last token. (The
        # window starting at the prefix needs one token more than we have, so a prefix never bans its own successor.)
        prefix = input_ids[:, cur_len + 1 - self.ngram_size :]
        windows = input_ids.unfold(dimension=1, size=self.ngram_size, step=1)
        matches = (windows[..., :-1] == prefix.unsqueeze(1)).all(dim=-1)

        # Non-matching windows go to a spare column past the vocab, where they can't unban another window's token
        vocab_size = scores.shape[-1]

View on GitHub (pinned to a597f97485)

Solutions

1. To disable n-gram blocking, remove no_repeat_ngram_size from the generate call / generation config
2. Otherwise pass a positive int, typically 2–4: no_repeat_ngram_size=2
3. Coerce external values: int(x) after asserting x >= 1

Example fix

# before
out = model.generate(**inputs, no_repeat_ngram_size=0)  # 'disable' -> ValueError

# after
out = model.generate(**inputs)  # disabled by omission
# or:
out = model.generate(**inputs, no_repeat_ngram_size=2)

def valid_ngram_size(n):
    return isinstance(n, int) and n > 0

def is_valid_ngram_size(n) -> bool:
    return type(n) is int and n > 0

try:
    proc = NoRepeatNGramLogitsProcessor(int(n))
except ValueError as e:
    raise ValueError(f'no_repeat_ngram_size={n!r} must be >= 1; omit to disable') from e

When it happens

Trigger: NoRepeatNGramLogitsProcessor(0); ngram_size=2.0 (float); model.generate(no_repeat_ngram_size=0) intending 'disabled'; numpy integers.

Common situations: Configs using 0 to disable the constraint (here you must omit the key); floats from templated configs; very small values like 1 producing degenerate outputs that loop back to invalid setups.

Related errors

• `temperature` (={temperature}) has to be a strictly positive
• `penalty` has to be a strictly positive float, but is {penal
• `prompt_ignore_length` has to be a positive integer, but is
• `top_p` has to be a float > 0 and < 1, but is {top_p}
• `min_tokens_to_keep` has to be a positive integer, but is {m

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/ca2094a9a4554c6a. Report an issue: GitHub.