huggingface/transformers · error · ValueError

`prefix_allowed_tokens_fn` returned an empty list for batch

Error message

`prefix_allowed_tokens_fn` returned an empty list for batch ID {batch_id}.This means that the constraint is unsatisfiable. Please check your implementationof `prefix_allowed_tokens_fn` 

What it means

Error "`prefix_allowed_tokens_fn` returned an empty list for batch ID {batch_id}.This means that the constraint is unsatisfiable. Please check your implementationof `prefix_allowed_tokens_fn` " thrown in huggingface/transformers.

Source

Thrown at src/transformers/generation/logits_process.py:1545

    Alice and Bob Marley
    ```
    """

    def __init__(self, prefix_allowed_tokens_fn: Callable[[int, torch.Tensor], list[int]], num_beams: int):
        self._prefix_allowed_tokens_fn = prefix_allowed_tokens_fn
        self._num_beams = num_beams

    @add_start_docstrings(LOGITS_PROCESSOR_INPUTS_DOCSTRING)
    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor) -> torch.FloatTensor:
        mask = torch.full_like(scores, -math.inf)
        batch_size = input_ids.shape[0] // self._num_beams

        for batch_id in range(batch_size):
            for beam_id in range(self._num_beams):
                sent = input_ids[batch_id * self._num_beams + beam_id]
                prefix_allowed_tokens = self._prefix_allowed_tokens_fn(batch_id, sent)
                if len(prefix_allowed_tokens) == 0:
                    raise ValueError(
                        f"`prefix_allowed_tokens_fn` returned an empty list for batch ID {batch_id}."
                        f"This means that the constraint is unsatisfiable. Please check your implementation"
                        f"of `prefix_allowed_tokens_fn` "
                    )
                mask[batch_id * self._num_beams + beam_id, prefix_allowed_tokens] = 0

        scores_processed = scores + mask
        return scores_processed


class ForcedBOSTokenLogitsProcessor(LogitsProcessor):
    r"""
    [`LogitsProcessor`] that enforces the specified token as the first generated token. Used with encoder-decoder
    models.

    Args:
        bos_token_id (`int`):
            The id of the token to force as the first generated token.

View on GitHub (pinned to a597f97485)

Solutions

1. Fix `prefix_allowed_tokens_fn` so it always returns at least one allowed token id for every prefix.
2. Include `eos_token_id` in the allowed set once generation may finish.

When it happens

Trigger: Raised in PrefixConstrainedLogitsProcessor.__call__ when the user-supplied prefix_allowed_tokens_fn returns an empty list for a batch element.

Common situations: A custom prefix_allowed_tokens_fn whose constraint becomes unsatisfiable at some decoding step, e.g. a trie walk reaching a dead end.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/a00271522ce8619c. Report an issue: GitHub.