huggingface/transformers · error · ValueError

`eta_cutoff` has to be a float > 0 and < 1, but is {epsilon}

Error message

`eta_cutoff` has to be a float > 0 and < 1, but is {epsilon}

What it means

Thrown by EtaLogitsWarper.__init__ when epsilon is <= 0 or >= 1. Eta sampling sets a dynamic probability floor min(epsilon, sqrt(epsilon)*exp(-entropy)), so the parameter must be strictly inside (0, 1). It is coerced with float() and stored as a torch tensor on the given device.

Source

Thrown at src/transformers/generation/logits_process.py:994

    A sequence: 1, 2, 3 | < 4 (left-hand pointer) ;
    <BLANKLINE>
    <BLANKLINE>

    >>> # With eta sampling, the output gets restricted to high-probability tokens. You can see it as a dynamic form of
    >>> # epsilon sampling that adapts its cutoff probability based on the entropy (high entropy = lower cutoff).
    >>> # Pro tip: The paper recommends using `eta_cutoff` values between 3e-4 to 4e-3
    >>> outputs = model.generate(**inputs, do_sample=True, eta_cutoff=0.1)
    >>> print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
    A sequence: 1, 2, 3, 4, 5, 6, 7, 8, 9
    ```
    """

    def __init__(
        self, epsilon: float, filter_value: float = -float("Inf"), min_tokens_to_keep: int = 1, device: str = "cpu"
    ):
        epsilon = float(epsilon)
        if epsilon <= 0 or epsilon >= 1:
            raise ValueError(f"`eta_cutoff` has to be a float > 0 and < 1, but is {epsilon}")

        min_tokens_to_keep = int(min_tokens_to_keep)
        if min_tokens_to_keep < 1:
            raise ValueError(
                f"`min_tokens_to_keep` has to be a strictly positive integer, but is {min_tokens_to_keep}"
            )

        self.epsilon = torch.tensor(epsilon, device=device)
        self.filter_value = filter_value
        self.min_tokens_to_keep = min_tokens_to_keep

    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        probabilities = scores.softmax(dim=-1)
        entropy = torch.distributions.Categorical(logits=scores).entropy()
        eta = torch.min(self.epsilon, torch.sqrt(self.epsilon) * torch.exp(-entropy))[..., None]
        indices_to_remove = probabilities < eta

View on GitHub (pinned to a597f97485)

Solutions

1. To disable eta sampling, remove eta_cutoff from the generate call / generation config
2. Otherwise pass a small float in (0, 1): eta_cutoff=3e-4
3. Validate 0 < eta_cutoff < 1 before generate()

Example fix

# before
out = model.generate(**inputs, do_sample=True, eta_cutoff=0)  # ValueError

# after
out = model.generate(**inputs, do_sample=True)  # disabled by omission
# or:
out = model.generate(**inputs, do_sample=True, eta_cutoff=3e-4)

def valid_eta(e):
    return isinstance(e, (int, float)) and 0.0 < float(e) < 1.0

def is_valid_eta(e) -> bool:
    return isinstance(e, (int, float)) and 0.0 < e < 1.0

try:
    proc = EtaLogitsWarper(float(e), device=inputs['input_ids'].device.type)
except ValueError as e:
    raise ValueError(f'eta_cutoff={e!r} must be in (0, 1); omit it to disable') from e

When it happens

Trigger: EtaLogitsWarper(0.0); eta_cutoff=1.0; model.generate(do_sample=True, eta_cutoff=0) intending 'disabled' (recommended practical range is 3e-4 to 4e-3).

Common situations: Using 0 as an 'off' sentinel (raises here — omit the parameter instead); configs ported from epsilon_cutoff with out-of-range values; percentage-style input.

Related errors

• `epsilon_cutoff` has to be a float > 0 and < 1, but is {epsi
• `temperature` (={temperature}) has to be a strictly positive
• `penalty` has to be a strictly positive float, but is {penal
• `prompt_ignore_length` has to be a positive integer, but is
• `top_p` has to be a float > 0 and < 1, but is {top_p}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/962119dcd15defb5. Report an issue: GitHub.