huggingface/transformers · error · ValueError

`max_new_tokens` must be greater than 0, but is

Error message

`max_new_tokens` must be greater than 0, but is {}.

What it means

GenerationConfig.validate() rejects max_new_tokens <= 0. Zero or negative generation length is meaningless for generate(), so it fails at validation time (which runs when generate() is called) rather than producing empty output silently.

Solutions

1. Pass a positive max_new_tokens
2. Guard budget computations: only call generate when the computed budget is >= 1
3. If you meant 'no new tokens', skip calling generate entirely

Example fix

# before
budget = max_length - input_ids.shape[1]  # can be <= 0
out = model.generate(input_ids, max_new_tokens=budget)
# after
budget = max_length - input_ids.shape[1]
out = model.generate(input_ids, max_new_tokens=budget) if budget > 0 else input_ids

def valid_max_new_tokens(v) -> bool:
    return v is None or (isinstance(v, int) and not isinstance(v, bool) and v > 0)

def is_positive_token_count(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v > 0

When it happens

Trigger: model.generate(max_new_tokens=0), GenerationConfig(max_new_tokens=-5), or computing max_new_tokens from a budget (max_length - prompt_len) that goes non-positive for long prompts.

Common situations: Programmatic budgets where prompt length meets/exceeds max_length; CLI/config-driven runs receiving 0 defaults; math like max(0, limit - len(prompt)) instead of skipping generation.

Related errors

• `early_stopping` must be a boolean or 'never', but is
• Invalid `cache_implementation
• `assistant_ensemble_weight` must be in the open interval…
• Invalid max_matching_ngram_size or num_output_tokens
• You provided `compile_config` as an instance of

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/1d0ffd97ea812566. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/generation/configuration_utils.py:670

        Note that some parameters not validated here are best validated at generate runtime, as they may depend on
        other inputs and/or the model, such as parameters related to the generation length.

        Args:
            strict (bool): If True, raise an exception for any issues found. If False, only log issues.
            user_set_attributes (set[str], *optional*): Names of attributes the caller explicitly provided. When
                supplied, "minor issue" warnings about conflicting flag combinations (e.g. sampling-only flags set
                while `do_sample=False`) only fire if the conflicting flag is in this set -- avoiding noisy warnings
                when the value was inherited from a model's default `generation_config.json`. When `None`, all set
                attributes are considered user-set (backward-compatible behavior for direct `validate()` calls).
        """
        minor_issues = {}  # format: {attribute_name: issue_description}

        # 1. Validation of individual attributes
        # 1.1. Decoding attributes
        if self.early_stopping not in {None, True, False, "never"}:
            raise ValueError(f"`early_stopping` must be a boolean or 'never', but is {self.early_stopping}.")
        if self.max_new_tokens is not None and self.max_new_tokens <= 0:
            raise ValueError(f"`max_new_tokens` must be greater than 0, but is {self.max_new_tokens}.")
        if self.assistant_ensemble_weight is not None and not (0.0 < self.assistant_ensemble_weight < 1.0):
            raise ValueError(
                f"`assistant_ensemble_weight` must be in the open interval `(0.0, 1.0)`, "
                f"but is {self.assistant_ensemble_weight}. Use `None` for standard (lossless) speculative decoding."
            )
        if self.pad_token_id is not None and self.pad_token_id < 0:
            minor_issues["pad_token_id"] = (
                f"`pad_token_id` should be positive but got {self.pad_token_id}. This will cause errors when batch "
                "generating, if there is padding. Please set `pad_token_id` explicitly as "
                "`model.generation_config.pad_token_id=PAD_TOKEN_ID` to avoid errors in generation"
            )
        # 1.2. Cache attributes
        # "paged" re-routes to continuous batching and so it is a valid cache implementation. But we do not want to test
        # it with the `generate` as the other would be, so we we cannot add it to ALL_CACHE_IMPLEMENTATIONS
        valid_cache_implementations = ALL_CACHE_IMPLEMENTATIONS + ("paged",)
        if self.cache_implementation is not None and self.cache_implementation not in valid_cache_implementations:
            raise ValueError(
                f"Invalid `cache_implementation` ({self.cache_implementation}). Choose one of: "

View on GitHub (pinned to a597f97485)