huggingface/transformers · error · ValueError

`typical_p` has to be a float > 0 and < 1, but is {mass}

Error message

`typical_p` has to be a float > 0 and < 1, but is {mass}

What it means

Thrown by the typical-decoding logits warper (TypicalLogitsWarper) __init__ when mass is not strictly between 0 and 1. Locally typical sampling keeps tokens whose deviation from conditional entropy is below a cumulative mass threshold; the constructor coerces with float(mass) first, so ints and numeric strings in (0,1) range are accepted, but 0.0 and 1.0 are rejected as degenerate.

Source

Thrown at src/transformers/generation/logits_process.py:836

    >>> # With `typical_p` set, the most obvious sequence is no longer produced, which may be good for your problem
    >>> set_seed(18)
    >>> outputs = model.generate(
    ...     **inputs, do_sample=True, typical_p=0.1, return_dict_in_generate=True, output_scores=True
    ... )
    >>> print(tokenizer.batch_decode(outputs.sequences, skip_special_tokens=True)[0])
    1, 2, 3 and 5

    >>> # We can see that the token corresponding to "4" (token 934) in the second position, the most likely token
    >>> # as seen with greedy decoding, was entirely blocked out
    >>> print(outputs.scores[1][0, 934])
    tensor(-inf)
    ```
    """

    def __init__(self, mass: float = 0.9, filter_value: float = -float("Inf"), min_tokens_to_keep: int = 1):
        mass = float(mass)
        if not (mass > 0 and mass < 1):
            raise ValueError(f"`typical_p` has to be a float > 0 and < 1, but is {mass}")
        if not isinstance(min_tokens_to_keep, int) or (min_tokens_to_keep < 1):
            raise ValueError(f"`min_tokens_to_keep` has to be a positive integer, but is {min_tokens_to_keep}")

        self.filter_value = filter_value
        self.mass = mass
        self.min_tokens_to_keep = min_tokens_to_keep

    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        # calculate entropy
        normalized = torch.nn.functional.log_softmax(scores, dim=-1)
        p = torch.exp(normalized)
        ent = -(normalized * p).nansum(-1, keepdim=True)

        # shift and sort
        shifted_scores = torch.abs((-normalized) - ent)
        sorted_scores, sorted_indices = torch.sort(shifted_scores, descending=False)
        sorted_logits = scores.gather(-1, sorted_indices)

View on GitHub (pinned to a597f97485)

Solutions

1. To disable typical filtering, remove typical_p from the generate call / generation config instead of setting it to 1.0
2. Otherwise pass a float strictly inside (0, 1): typical_p=0.9
3. Clamp/validate config values: 0.0 < typical_p < 1.0

Example fix

# before
out = model.generate(**inputs, do_sample=True, typical_p=1.0)  # 'disable' intent -> ValueError

# after
out = model.generate(**inputs, do_sample=True)  # typical_p omitted = disabled
# or a valid value:
out = model.generate(**inputs, do_sample=True, typical_p=0.9)

def valid_typical_p(m):
    return isinstance(m, (int, float)) and 0.0 < float(m) < 1.0

def is_valid_typical_p(m) -> bool:
    return isinstance(m, (int, float)) and 0.0 < m < 1.0

try:
    proc = TypicalLogitsWarper(float(m))
except ValueError as e:
    raise ValueError(f'typical_p={m!r} must be strictly inside (0, 1); omit it to disable') from e

When it happens

Trigger: TypicalLogitsWarper(mass=1.0) expecting 'keep everything'; mass=0.0; mass=1.5; model.generate(do_sample=True, typical_p=1.0) — note generation's typical_p=1.0 routes here and raises, unlike top_p where 1.0 is allowed.

Common situations: Users coming from top_p semantics where 1.0 disables the filter; generation configs saved with typical_p: 1.0 as a placeholder for 'off'; percentage-style values like 95.

Related errors

• `temperature` (={temperature}) has to be a strictly positive
• `penalty` has to be a strictly positive float, but is {penal
• `prompt_ignore_length` has to be a positive integer, but is
• `top_p` has to be a float > 0 and < 1, but is {top_p}
• `min_tokens_to_keep` has to be a positive integer, but is {m

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/981370ee5e8e0253. Report an issue: GitHub.