huggingface/transformers · error · ValueError

Some generation parameters are set in the model config. Thes

Error message

Some generation parameters are set in the model config. These should go into `model.generation_config`as opposed to `model.config`. 
Generation parameters found: {str(generation_parameters)}

What it means

Raised by PreTrainedConfig.save_pretrained (and model save_pretrained which serializes the config) when generation-related attributes (e.g. temperature, top_k, max_length) are found on model.config. Since Transformers v5, generation parameters must live on model.generation_config; the config file itself is required to be free of them. The error lists exactly which parameters leaked into the config.

Source

Thrown at src/transformers/configuration_utils.py:574

        Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the
        [`~PreTrainedConfig.from_pretrained`] class method.

        Args:
            save_directory (`str` or `os.PathLike`):
                Directory where the configuration JSON file will be saved (will be created if it does not exist).
            push_to_hub (`bool`, *optional*, defaults to `False`):
                Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the
                repository you want to push to with `repo_id` (will default to the name of `save_directory` in your
                namespace).
            kwargs (`dict[str, Any]`, *optional*):
                Additional key word arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method.
        """
        if os.path.isfile(save_directory):
            raise AssertionError(f"Provided path ({save_directory}) should be a directory, not a file")

        generation_parameters = self._get_generation_parameters()
        if len(generation_parameters) > 0:
            raise ValueError(
                "Some generation parameters are set in the model config. These should go into `model.generation_config`"
                f"as opposed to `model.config`. \nGeneration parameters found: {str(generation_parameters)}",
            )

        os.makedirs(save_directory, exist_ok=True)

        if push_to_hub:
            commit_message = kwargs.pop("commit_message", None)
            repo_id = kwargs.pop("repo_id", save_directory.split(os.path.sep)[-1])
            repo_id = hf_api().create_repo(repo_id, exist_ok=True, **kwargs).repo_id
            files_timestamps = self._get_files_timestamps(save_directory)

        # This attribute is important to know on load, but should not be serialized on save.
        if "transformers_weights" in self:
            delattr(self, "transformers_weights")

        # If we have a custom config, we copy the file defining it in the folder and set the attributes so it can be
        # loaded from the Hub.

View on GitHub (pinned to a597f97485)

Solutions

Move the offending parameters to model.generation_config (e.g. model.generation_config.temperature = 0.7) and delete them from config (delattr(config, 'temperature')).
When loading old checkpoints, set the generation params via AutoConfig/AutoModelForCausalLM.from_pretrained(..., generation_config=GenerationConfig(...)) instead of through config kwargs.
Regenerate the config.json of legacy checkpoints: load it, strip generation keys listed in the error message, and re-save.

Example fix

// before
config = GPT2Config(temperature=0.7, top_k=50)
model.save_pretrained("out")  # ValueError

// after
config = GPT2Config()
model.save_pretrained("out", generation_config=GenerationConfig(temperature=0.7, top_k=50))

Defensive patterns

Strategy: validation

Validate before calling

from transformers import GenerationConfig
GEN_PARAMS = {"temperature", "top_k", "top_p", "max_length", "do_sample", "num_beams"}
leaked = [k for k in model.config.to_dict() if k in GenerationConfig().to_dict() and k not in ("transformers_version",)]
if leaked:
    for k in leaked:
        setattr(model.generation_config, k, getattr(model.config, k))
        delattr(model.config, k)

Type guard

def config_is_generation_clean(config) -> bool:
    return len(config._get_generation_parameters()) == 0

Try / catch

try:
    model.save_pretrained(out_dir)
except ValueError as e:
    if "generation parameters" in str(e).lower():
        # move listed params to model.generation_config and retry
        raise

Prevention

Never set generation kwargs through model config constructors; always pass generation_config to save_pretrained or assign to model.generation_config.
Add a CI check that loads and re-saves canonical checkpoints with the current Transformers major version.

When it happens

Trigger: Calling config.save_pretrained(...) or model.save_pretrained(...) / model.push_to_hub() on a config that has generation kwargs set, e.g. GPT2Config(temperature=0.7, top_k=50) or loading an old checkpoint whose config.json still contains generation keys and re-saving it.

Common situations: Migrating a repo or checkpoint from Transformers v4 to v5 where old config.json files carry generation fields; passing from_pretrained(..., temperature=...) style kwargs that end up on config; code that sets model.config.max_length directly.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/95a9b722c1fdf744. Report an issue: GitHub.