huggingface/transformers · error · ValueError

`top_h` must be in the range (0, 1].

Error message

`top_h` must be in the range (0, 1].

What it means

Thrown by TopHLogitsProcessor.__init__ when top_h is not in the open-closed interval (0, 1]. Top-H sampling filters tokens by an entropy-based threshold scaled by top_h, so it must be a fraction greater than 0 and at most 1 (1.0 disables the filter, 0 would keep nothing).

Source

Thrown at src/transformers/generation/logits_process.py:639

    >>> from transformers import AutoTokenizer, AutoModelForCausalLM

    >>> model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
    >>> tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")

    >>> inputs = tokenizer("A sequence: 1, 2", return_tensors="pt")

    >>> outputs = model.generate(**inputs, do_sample=True, top_h=0.4)
    >>> print(tokenizer.batch_decode(outputs, skip_special_tokens=True)[0])
    A sequence: 1, 2, 3, 4, 5, 6, 7, 8, 9
    ```
    """

    def __init__(self, top_h: float, filter_value: float = -float("Inf")):
        super().__init__()

        # input checks
        if not (0 < top_h <= 1):
            raise ValueError("`top_h` must be in the range (0, 1].")

        # Maximum number of top tokens to consider before applying the entropy-based filter.
        # Acts as a cap for efficiency and numerical stability — increasing this allows more
        # tokens to be evaluated but may slow down generation. Default is 100.
        self.top_n = 100

        self.top_h = top_h
        self.filter_value = filter_value

    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        """
        Filters logits using Top-H sampling.

        Args:
            input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`):
                Input token IDs.
            scores (`torch.FloatTensor` of shape `(batch_size, vocab_size)`):
                Raw logits from the model.

View on GitHub (pinned to a597f97485)

Solutions

1. Use a fraction in (0, 1]: top_h=0.4 (the docstring example)
2. Convert percentages: top_h = pct / 100.0 and clamp to at most 1.0
3. Validate external config values with 0 < top_h <= 1 before generate()

Example fix

# before
proc = TopHLogitsProcessor(0)  # ValueError

# after
proc = TopHLogitsProcessor(0.4)
out = model.generate(**inputs, do_sample=True, top_h=0.4)

def valid_top_h(h):
    return isinstance(h, (int, float)) and 0 < float(h) <= 1

def is_valid_top_h(h) -> bool:
    return isinstance(h, (int, float)) and 0.0 < h <= 1.0

try:
    proc = TopHLogitsProcessor(float(h))
except ValueError as e:
    raise ValueError(f'top_h={h!r} must be in (0, 1]') from e

When it happens

Trigger: TopHLogitsProcessor(0.0); top_h=1.1; top_h=-0.2; model.generate(do_sample=True, top_h=0) via generation config.

Common situations: Newer parameter with fewer examples in the wild — values copied from top_p configs where 0 was tolerated; percentage-style input (40 instead of 0.4); sweeps crossing the boundary.

Related errors

• `temperature` (={temperature}) has to be a strictly positive
• `penalty` has to be a strictly positive float, but is {penal
• `prompt_ignore_length` has to be a positive integer, but is
• `top_p` has to be a float > 0 and < 1, but is {top_p}
• `min_tokens_to_keep` has to be a positive integer, but is {m

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d5fb1b7d01b61c27. Report an issue: GitHub.