huggingface/transformers · error · ValueError

Input length of {input_ids_string} is {input_ids_length}, bu

Error message

Input length of {input_ids_string} is {input_ids_length}, but `max_length` is set to {generation_config.max_length}. This can lead to unexpected behavior. You should consider increasing `max_length` or, better yet, setting `max_new_tokens`.

What it means

Generation counts total length: prompt plus new tokens must fit within `max_length`. If `input_ids` (or `decoder_input_ids`) is already at least `max_length` tokens long, there is no room to generate anything, so `generate` raises rather than returning an empty/invalid continuation.

Source

Thrown at src/transformers/generation/utils.py:1684

                " generate arguments will also show up in this list)"
            )

    def _validate_generated_length(
        self: "GenerativePreTrainedModel", generation_config, input_ids_length, has_default_max_length
    ):
        """Performs validation related to the resulting generated length"""
        # 1. Max length warnings related to poor parameterization
        if has_default_max_length and generation_config.max_new_tokens is None:
            # 20 is the default max_length of the generation config
            warnings.warn(
                f"Using the model-agnostic default `max_length` (={generation_config.max_length}) to control the "
                "generation length. We recommend setting `max_new_tokens` to control the maximum length of the "
                "generation.",
                UserWarning,
            )
        if input_ids_length >= generation_config.max_length:
            input_ids_string = "decoder_input_ids" if self.config.is_encoder_decoder else "input_ids"
            raise ValueError(
                f"Input length of {input_ids_string} is {input_ids_length}, but `max_length` is set to"
                f" {generation_config.max_length}. This can lead to unexpected behavior. You should consider"
                " increasing `max_length` or, better yet, setting `max_new_tokens`."
            )

        # 2. Min length warnings due to unfeasible parameter combinations
        min_length_error_suffix = (
            " Generation will stop at the defined maximum length. You should decrease the minimum length and/or "
            "increase the maximum length."
        )
        if has_default_max_length:
            min_length_error_suffix += (
                f" Note that `max_length` is set to {generation_config.max_length}, its default value."
            )
        if generation_config.min_length is not None and generation_config.min_length > generation_config.max_length:
            warnings.warn(
                f"Unfeasible length constraints: `min_length` ({generation_config.min_length}) is larger than"
                f" the maximum possible length ({generation_config.max_length})." + min_length_error_suffix,

View on GitHub (pinned to a597f97485)

Solutions

1. Prefer `max_new_tokens`: `model.generate(**inputs, max_new_tokens=200)` — it is computed relative to the prompt length.
2. Or raise `max_length` above prompt+desired output: `model.generate(**inputs, max_length=input_len + 200)`.
3. Or truncate the prompt at tokenization: `tokenizer(text, truncation=True, max_length=...)`.
4. For encoder-decoder models, keep the decoder start short and set `max_new_tokens`.

Example fix

# before
out = model.generate(**tokenizer(long_doc, return_tensors="pt"))  # prompt >= max_length(20) -> ValueError

# after
out = model.generate(**tokenizer(long_doc, return_tensors="pt", truncation=True, max_length=1024), max_new_tokens=256)

prompt_len = inputs["input_ids"].shape[-1]
max_len = kwargs.get("max_length", model.generation_config.max_length)
if max_len is not None and prompt_len >= max_len:
    kwargs["max_new_tokens"] = kwargs.get("max_new_tokens", 64)  # or raise/increase max_length

When it happens

Trigger: `model.generate(**inputs)` with default `max_length=20` and a prompt >= 20 tokens; a long prompt with a small saved `max_length`; encoder-decoder models where `decoder_input_ids` length >= `max_length`.

Common situations: Forgetting `max_new_tokens` so the legacy default `max_length=20` applies (a related UserWarning also fires); truncation disabled in the tokenizer (`truncation=False`) so long documents exceed the limit; configs where `max_length` was tuned for short prompts.

Related errors

• Error with input length {len(input_ids)} vs {batch_length}
• `crop` was called, but the current layer does not track past
• Once the sliding window size has been reached, `DynamicSlidi
• `crop` was called, but the current layer does not track past
• Some generation parameters are set in the model config. Thes

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f17a6557af2c8c89. Report an issue: GitHub.