{"record":{"id":"981370ee5e8e0253","repo":"huggingface/transformers","slug":"typical-p-has-to-be-a-float-0-and-1-but-is","errorCode":null,"errorMessage":"`typical_p` has to be a float > 0 and < 1, but is {mass}","messagePattern":"`typical_p` has to be a float > 0 and < 1, but is (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/logits_process.py","lineNumber":836,"sourceCode":"    >>> # With `typical_p` set, the most obvious sequence is no longer produced, which may be good for your problem\n    >>> set_seed(18)\n    >>> outputs = model.generate(\n    ...     **inputs, do_sample=True, typical_p=0.1, return_dict_in_generate=True, output_scores=True\n    ... )\n    >>> print(tokenizer.batch_decode(outputs.sequences, skip_special_tokens=True)[0])\n    1, 2, 3 and 5\n\n    >>> # We can see that the token corresponding to \"4\" (token 934) in the second position, the most likely token\n    >>> # as seen with greedy decoding, was entirely blocked out\n    >>> print(outputs.scores[1][0, 934])\n    tensor(-inf)\n    ```\n    \"\"\"\n\n    def __init__(self, mass: float = 0.9, filter_value: float = -float(\"Inf\"), min_tokens_to_keep: int = 1):\n        mass = float(mass)\n        if not (mass > 0 and mass < 1):\n            raise ValueError(f\"`typical_p` has to be a float > 0 and < 1, but is {mass}\")\n        if not isinstance(min_tokens_to_keep, int) or (min_tokens_to_keep < 1):\n            raise ValueError(f\"`min_tokens_to_keep` has to be a positive integer, but is {min_tokens_to_keep}\")\n\n        self.filter_value = filter_value\n        self.mass = mass\n        self.min_tokens_to_keep = min_tokens_to_keep\n\n    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)\n    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:\n        # calculate entropy\n        normalized = torch.nn.functional.log_softmax(scores, dim=-1)\n        p = torch.exp(normalized)\n        ent = -(normalized * p).nansum(-1, keepdim=True)\n\n        # shift and sort\n        shifted_scores = torch.abs((-normalized) - ent)\n        sorted_scores, sorted_indices = torch.sort(shifted_scores, descending=False)\n        sorted_logits = scores.gather(-1, sorted_indices)","sourceCodeStart":818,"sourceCodeEnd":854,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/logits_process.py#L818-L854","documentation":"Thrown by the typical-decoding logits warper (TypicalLogitsWarper) __init__ when mass is not strictly between 0 and 1. Locally typical sampling keeps tokens whose deviation from conditional entropy is below a cumulative mass threshold; the constructor coerces with float(mass) first, so ints and numeric strings in (0,1) range are accepted, but 0.0 and 1.0 are rejected as degenerate.","triggerScenarios":"TypicalLogitsWarper(mass=1.0) expecting 'keep everything'; mass=0.0; mass=1.5; model.generate(do_sample=True, typical_p=1.0) — note generation's typical_p=1.0 routes here and raises, unlike top_p where 1.0 is allowed.","commonSituations":"Users coming from top_p semantics where 1.0 disables the filter; generation configs saved with typical_p: 1.0 as a placeholder for 'off'; percentage-style values like 95.","solutions":["To disable typical filtering, remove typical_p from the generate call / generation config instead of setting it to 1.0","Otherwise pass a float strictly inside (0, 1): typical_p=0.9","Clamp/validate config values: 0.0 < typical_p < 1.0"],"exampleFix":"# before\nout = model.generate(**inputs, do_sample=True, typical_p=1.0)  # 'disable' intent -> ValueError\n\n# after\nout = model.generate(**inputs, do_sample=True)  # typical_p omitted = disabled\n# or a valid value:\nout = model.generate(**inputs, do_sample=True, typical_p=0.9)","handlingStrategy":"validation","validationCode":"def valid_typical_p(m):\n    return isinstance(m, (int, float)) and 0.0 < float(m) < 1.0","typeGuard":"def is_valid_typical_p(m) -> bool:\n    return isinstance(m, (int, float)) and 0.0 < m < 1.0","tryCatchPattern":"try:\n    proc = TypicalLogitsWarper(float(m))\nexcept ValueError as e:\n    raise ValueError(f'typical_p={m!r} must be strictly inside (0, 1); omit it to disable') from e","preventionTips":["typical_p=1.0 raises here, unlike top_p — omit the parameter to disable","Default/typical value is 0.9","Validate strict inequality when loading generation configs"],"tags":["generation","typical-sampling","typical-p","argument-validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}