huggingface/transformers · error · ValueError

When generating with token healing, you must pass the model

Error message

 When generating with token healing, you must pass the model's tokenizer to the `tokenizer` argument of `generate`.

What it means

Error " When generating with token healing, you must pass the model's tokenizer to the `tokenizer` argument of `generate`." thrown in huggingface/transformers.

Source

Thrown at src/transformers/generation/utils.py:2710

            if this_peer_finished_flag.item() == 0.0:
                return False
        elif this_peer_finished:
            return False
        return True

    def heal_tokens(
        self, input_ids: torch.LongTensor, tokenizer: Optional["PreTrainedTokenizerBase"] = None
    ) -> torch.LongTensor:
        r"""
        Generates sequences of token ids for models with a language modeling head.
        Parameters:
            input_ids (`torch.LongTensor`): The sequence used as a prompt for the generation.
            tokenizer (`PreTrainedTokenizerBase`, *optional*): The tokenizer used to decode the input ids.
        Return:
            `torch.LongTensor` where each sequence has its tail token replaced with its appropriate extension.
        """
        if tokenizer is None:
            raise ValueError(
                " When generating with token healing, you must pass the model's tokenizer to the `tokenizer` "
                "argument of `generate`."
            )

        bos_token_id, pad_token_id = tokenizer.bos_token_id, tokenizer.pad_token_id
        vocab_trie = ExtensionsTrie(tokenizer.get_vocab())
        generation_config = GenerationConfig(max_new_tokens=1, pad_token_id=pad_token_id)

        # assumption: leading/trailing whitespace is not meaningful, so the prompts are
        # stripped before re-tokenizing to desensitize generation to whitespace artefacts
        prompts = [p.strip() for p in tokenizer.decode(input_ids, skip_special_tokens=True)]
        input_ids = tokenizer(
            prompts,
            return_tensors="pt",
            padding=True,
        ).input_ids.to(input_ids.device)

        # replace bos with pad to not condition healing on it

View on GitHub (pinned to a597f97485)

Solutions

1. Pass the model's tokenizer via `generate(..., tokenizer=tokenizer)` when using token healing.
2. Disable token healing if no tokenizer is available.

When it happens

Trigger: Raised in generate() when token healing is enabled but no tokenizer was passed to generate().

Common situations: Using token healing (heal_tokens / related flag) without forwarding the model tokenizer into generate().

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/769149d163aee157. Report an issue: GitHub.