huggingface/transformers · error · ValueError

`max_new_tokens` must be greater than 0, but is {}.

Error message

`max_new_tokens` must be greater than 0, but is {}.

What it means

GenerationConfig.validate() rejects max_new_tokens <= 0. Zero or negative generation length is meaningless for generate(), so it fails at validation time (which runs when generate() is called) rather than producing empty output silently.

Source

Thrown at src/transformers/generation/configuration_utils.py:670

        Note that some parameters not validated here are best validated at generate runtime, as they may depend on
        other inputs and/or the model, such as parameters related to the generation length.

        Args:
            strict (bool): If True, raise an exception for any issues found. If False, only log issues.
            user_set_attributes (set[str], *optional*): Names of attributes the caller explicitly provided. When
                supplied, "minor issue" warnings about conflicting flag combinations (e.g. sampling-only flags set
                while `do_sample=False`) only fire if the conflicting flag is in this set -- avoiding noisy warnings
                when the value was inherited from a model's default `generation_config.json`. When `None`, all set
                attributes are considered user-set (backward-compatible behavior for direct `validate()` calls).
        """
        minor_issues = {}  # format: {attribute_name: issue_description}

        # 1. Validation of individual attributes
        # 1.1. Decoding attributes
        if self.early_stopping not in {None, True, False, "never"}:
            raise ValueError(f"`early_stopping` must be a boolean or 'never', but is {self.early_stopping}.")
        if self.max_new_tokens is not None and self.max_new_tokens <= 0:
            raise ValueError(f"`max_new_tokens` must be greater than 0, but is {self.max_new_tokens}.")
        if self.assistant_ensemble_weight is not None and not (0.0 < self.assistant_ensemble_weight < 1.0):
            raise ValueError(
                f"`assistant_ensemble_weight` must be in the open interval `(0.0, 1.0)`, "
                f"but is {self.assistant_ensemble_weight}. Use `None` for standard (lossless) speculative decoding."
            )
        if self.pad_token_id is not None and self.pad_token_id < 0:
            minor_issues["pad_token_id"] = (
                f"`pad_token_id` should be positive but got {self.pad_token_id}. This will cause errors when batch "
                "generating, if there is padding. Please set `pad_token_id` explicitly as "
                "`model.generation_config.pad_token_id=PAD_TOKEN_ID` to avoid errors in generation"
            )
        # 1.2. Cache attributes
        # "paged" re-routes to continuous batching and so it is a valid cache implementation. But we do not want to test
        # it with the `generate` as the other would be, so we we cannot add it to ALL_CACHE_IMPLEMENTATIONS
        valid_cache_implementations = ALL_CACHE_IMPLEMENTATIONS + ("paged",)
        if self.cache_implementation is not None and self.cache_implementation not in valid_cache_implementations:
            raise ValueError(
                f"Invalid `cache_implementation` ({self.cache_implementation}). Choose one of: "

View on GitHub (pinned to a597f97485)

Solutions

1. Pass a positive max_new_tokens
2. Guard budget computations: only call generate when the computed budget is >= 1
3. If you meant 'no new tokens', skip calling generate entirely

Example fix

# before
budget = max_length - input_ids.shape[1]  # can be <= 0
out = model.generate(input_ids, max_new_tokens=budget)
# after
budget = max_length - input_ids.shape[1]
out = model.generate(input_ids, max_new_tokens=budget) if budget > 0 else input_ids

def valid_max_new_tokens(v) -> bool:
    return v is None or (isinstance(v, int) and not isinstance(v, bool) and v > 0)

def is_positive_token_count(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v > 0

When it happens

Trigger: model.generate(max_new_tokens=0), GenerationConfig(max_new_tokens=-5), or computing max_new_tokens from a budget (max_length - prompt_len) that goes non-positive for long prompts.

Common situations: Programmatic budgets where prompt length meets/exceeds max_length; CLI/config-driven runs receiving 0 defaults; math like max(0, limit - len(prompt)) instead of skipping generation.

Related errors

• `early_stopping` must be a boolean or 'never', but is {}.
• Invalid `cache_implementation` ({}). Choose one of: {}
• Invalid max_matching_ngram_size or num_output_tokens
• `assistant_ensemble_weight` must be in the open interval `(0
• You provided `compile_config` as an instance of {}, but it m

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/1d0ffd97ea812566. Report an issue: GitHub.