huggingface/transformers · error · ValueError

There are one or more stop strings, either in the arguments

Error message

There are one or more stop strings, either in the arguments to `generate` or in the model's generation config, but we could not locate a tokenizer. When generating with stop strings, you must pass the model's tokenizer to the `tokenizer` argument of `generate`.

What it means

Stopping criteria can include stop STRINGS, which must be matched against decoded text and therefore require a tokenizer. `generate` tries to build a `StopStringCriteria` from `generation_config.stop_strings`, and if no tokenizer was passed (and none could be inferred) it raises instead of silently ignoring your stop strings.

Source

Thrown at src/transformers/generation/utils.py:1377

        self: "GenerativePreTrainedModel",
        generation_config: GenerationConfig,
        stopping_criteria: StoppingCriteriaList | None,
        tokenizer: Optional["PreTrainedTokenizerBase"] = None,
    ) -> StoppingCriteriaList:
        criteria = StoppingCriteriaList()
        if generation_config.max_length is not None:
            max_position_embeddings = getattr(self.config, "max_position_embeddings", None)
            criteria.append(
                MaxLengthCriteria(
                    max_length=generation_config.max_length,
                    max_position_embeddings=max_position_embeddings,
                )
            )
        if generation_config.max_time is not None:
            criteria.append(MaxTimeCriteria(max_time=generation_config.max_time))
        if generation_config.stop_strings is not None:
            if tokenizer is None:
                raise ValueError(
                    "There are one or more stop strings, either in the arguments to `generate` or in the "
                    "model's generation config, but we could not locate a tokenizer. When generating with "
                    "stop strings, you must pass the model's tokenizer to the `tokenizer` argument of `generate`."
                )
            criteria.append(StopStringCriteria(stop_strings=generation_config.stop_strings, tokenizer=tokenizer))
        if generation_config._eos_token_tensor is not None:
            criteria.append(EosTokenCriteria(eos_token_id=generation_config._eos_token_tensor))
        if (
            generation_config.is_assistant
            and generation_config.assistant_confidence_threshold is not None
            and generation_config.assistant_confidence_threshold > 0
        ):
            criteria.append(
                ConfidenceCriteria(assistant_confidence_threshold=generation_config.assistant_confidence_threshold)
            )
        criteria = self._merge_criteria_processor_list(criteria, stopping_criteria)
        return criteria

View on GitHub (pinned to a597f97485)

Solutions

Pass the tokenizer to generate: `model.generate(**tokenizer(prompt, return_tensors="pt"), stop_strings=["\n\n"], tokenizer=tokenizer)`.
If you tokenize yourself, still pass `tokenizer=tokenizer` alongside `input_ids`.
If stop strings are unwanted, remove `stop_strings` from `model.generation_config` (`model.generation_config.stop_strings = None`) or from your generate kwargs.

Example fix

# before
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
out = model.generate(input_ids, stop_strings=["User:"])  # ValueError: no tokenizer

# after
out = model.generate(
    **tokenizer(prompt, return_tensors="pt"),
    stop_strings=["User:"],
    tokenizer=tokenizer,
)

Defensive patterns

Strategy: validation

Validate before calling

if generation_config.stop_strings and tokenizer is None:
    raise ValueError("stop_strings requires passing `tokenizer` to generate()")

Prevention

Always pass the full tokenizer output plus `tokenizer=tokenizer` when using stop_strings.
When building generate kwargs programmatically, add tokenizer automatically if `stop_strings` is present.
Prefer stop_strings + tokenizer over manual post-hoc string stopping so the model stops at the right step.

When it happens

Trigger: `generation_config.stop_strings=["\n\n"]` (set in `generate(...)` kwargs or in the model's `generation_config.json`) while calling `model.generate(input_ids, ...)` without `tokenizer=...` — common when inputs are prepared manually instead of via `pipeline` or `model.generate(**tokenizer_inputs)`.

Common situations: Using pre-tokenized `input_ids` tensors; models without an attached `tokenizer` attribute; copying a `generation_config.json` from the Hub that ships `stop_strings`; calling a served/wrapped model where only ids are forwarded.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/5a42048337844fcc. Report an issue: GitHub.