What causes "`top_k` has to be a strictly positive integer, but is {top_k}" in huggingface/transformers?

To disable top-k filtering, remove top_k from the generate call / generation config entirely

huggingface/transformers · error · ValueError

`top_k` has to be a strictly positive integer, but is {top_k

Error message

`top_k` has to be a strictly positive integer, but is {top_k}

What it means

Thrown by TopKLogitsWarper.__init__ when top_k is not a Python int or is <= 0. Top-k sampling keeps only the k highest logits, so k must be a positive integer count. Note self.top_k = max(top_k, min_tokens_to_keep) is stored, and the __call__ clamps against vocab size at runtime, so only the constructor check can fail.

Source

Thrown at src/transformers/generation/logits_process.py:583

    >>> # With sampling, the output is unexpected -- sometimes too unexpected.
    >>> outputs = model.generate(**inputs, do_sample=True)
    >>> print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
    A sequence: A, B, C, D, E — S — O, P — R

    >>> # With `top_k` sampling, the output gets restricted the k most likely tokens.
    >>> # Pro tip: In practice, LLMs use `top_k` in the 5-50 range.
    >>> outputs = model.generate(**inputs, do_sample=True, top_k=2)
    >>> print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
    A sequence: A, B, C, D, E, F, G, H, I
    ```
    """

    supports_continuous_batching = True

    def __init__(self, top_k: int, filter_value: float = -float("Inf"), min_tokens_to_keep: int = 1):
        if not isinstance(top_k, int) or top_k <= 0:
            raise ValueError(f"`top_k` has to be a strictly positive integer, but is {top_k}")

        self.top_k = max(top_k, min_tokens_to_keep)
        self.filter_value = filter_value
        self.min_tokens_to_keep = min_tokens_to_keep  # used for CB processor initialization

    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        top_k = min(self.top_k, scores.size(-1))  # Safety check
        # Remove all tokens with a probability less than the last token of the top-k
        indices_to_remove = scores < torch.topk(scores, top_k)[0][..., -1, None]
        scores_processed = scores.masked_fill(indices_to_remove, self.filter_value)
        return scores_processed


class TopHLogitsWarper(LogitsProcessor):
    """
    [`LogitsProcessor`] that implements Top-H sampling, a decoding method which adaptively selects a subset of
    high-probability tokens based on entropy and cumulative probability constraints.

View on GitHub (pinned to a597f97485)

Solutions

To disable top-k filtering, remove top_k from the generate call / generation config entirely
Otherwise pass a positive int: top_k=50
Coerce external values: int(top_k) after checking top_k >= 1

Example fix

# before
out = model.generate(**inputs, do_sample=True, top_k=0)  # 'disable' convention -> ValueError

# after
out = model.generate(**inputs, do_sample=True)  # top_k omitted = disabled
# or a large k to approximate disabled:
out = model.generate(**inputs, do_sample=True, top_k=model.config.vocab_size)

Defensive patterns

Strategy: validation

Validate before calling

def valid_top_k(k):
    return isinstance(k, int) and k > 0

Type guard

def is_valid_top_k(k) -> bool:
    return type(k) is int and k > 0

Try / catch

try:
    proc = TopKLogitsWarper(int(k))
except ValueError as e:
    raise ValueError(f'top_k={k!r} must be a positive int; omit it to disable') from e

Prevention

top_k=0 does not mean 'disabled' here — omit the parameter instead
Pass ints, not floats or numpy scalars
Common LLM range is 5–50

When it happens

Trigger: TopKLogitsWarper(0); top_k=-5; top_k=50.0 (float); model.generate(do_sample=True, top_k=0) intending 'disabled' — this library requires omitting top_k or setting it to the vocab size instead.

Common situations: Configs where top_k: 0 conventionally means 'off' (as in some other inference stacks) — here it raises; passing top_k as float from a config; numpy ints from sweep grids.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/bfdbf1ce7ac911f0. Report an issue: GitHub.