huggingface/transformers · error · ValueError

`top_p` has to be a float > 0 and < 1, but is {top_p}

Error message

`top_p` has to be a float > 0 and < 1, but is {top_p}

What it means

Thrown by TopPLogitsWarper.__init__ when top_p is < 0 or > 1.0 after a float() coercion. Top-p (nucleus) sampling keeps the smallest set of tokens whose cumulative probability exceeds top_p, so the threshold must be a probability. Note the code coerces with float(top_p) first, so numeric strings and ints are accepted; the boundary values 0.0 and 1.0 also pass despite the message saying '> 0 and < 1' — 1.0 is effectively a no-op.

Source

Thrown at src/transformers/generation/logits_process.py:518

    >>> print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
    A sequence: 1, 2, 3 | < 4 (left-hand pointer) ;
    <BLANKLINE>
    <BLANKLINE>

    >>> # With `top_p` sampling, the output gets restricted to high-probability tokens.
    >>> # Pro tip: In practice, LLMs use `top_p` in the 0.9-0.95 range.
    >>> outputs = model.generate(**inputs, do_sample=True, top_p=0.1)
    >>> print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
    A sequence: 1, 2, 3, 4, 5, 6, 7, 8, 9
    ```
    """

    supports_continuous_batching = True

    def __init__(self, top_p: float, filter_value: float = -float("Inf"), min_tokens_to_keep: int = 1):
        top_p = float(top_p)
        if top_p < 0 or top_p > 1.0:
            raise ValueError(f"`top_p` has to be a float > 0 and < 1, but is {top_p}")
        if not isinstance(min_tokens_to_keep, int) or (min_tokens_to_keep < 1):
            raise ValueError(f"`min_tokens_to_keep` has to be a positive integer, but is {min_tokens_to_keep}")

        self.top_p = top_p
        self.filter_value = filter_value
        self.min_tokens_to_keep = min_tokens_to_keep

    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        sorted_logits, sorted_indices = torch.sort(scores, descending=False)
        cumulative_probs = sorted_logits.softmax(dim=-1).cumsum(dim=-1)

        # Remove tokens with cumulative top_p above the threshold (token with 0 are kept)
        sorted_indices_to_remove = cumulative_probs <= (1 - self.top_p)
        # Keep at least min_tokens_to_keep
        sorted_indices_to_remove[..., -self.min_tokens_to_keep :] = 0

        # scatter sorted tensors to original indexing

View on GitHub (pinned to a597f97485)

Solutions

  1. Clamp or correct the value to [0, 1]: top_p=0.9
  2. If the value came as a percentage (e.g. 90), divide by 100 before use
  3. Validate sweep/grid values: assert 0.0 <= top_p <= 1.0 before generate()

Example fix

# before
out = model.generate(**inputs, do_sample=True, top_p=90)  # percent mistake -> ValueError

# after
top_p = min(max(top_p_raw / 100.0, 0.0), 1.0) if top_p_raw > 1 else top_p_raw
out = model.generate(**inputs, do_sample=True, top_p=top_p)
Defensive patterns

Strategy: validation

Validate before calling

def valid_top_p(p):
    try:
        p = float(p)
    except (TypeError, ValueError):
        return False
    return 0.0 <= p <= 1.0

Type guard

def is_valid_top_p(p) -> bool:
    try:
        return 0.0 <= float(p) <= 1.0
    except (TypeError, ValueError):
        return False

Try / catch

try:
    proc = TopPLogitsWarper(float(top_p))
except ValueError as e:
    raise ValueError(f'top_p={top_p!r} must be in [0, 1]') from e

Prevention

When it happens

Trigger: TopPLogitsWarper(1.5); top_p=-0.1; model.generate(do_sample=True, top_p=1.2) via a typo'd generation config; hyperparameter sweeps stepping past 1.0.

Common situations: Generation-config JSON/YAML with top_p mistyped as 1.5; sweeps written as numpy floats > 1; confusing top_p with a percentage (passing 90 instead of 0.9).

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/dd3b191459432740. Report an issue: GitHub.