huggingface/transformers · error · ValueError

Greedy methods (do_sample != True) without beam search do…

Error message

Greedy methods (do_sample != True) without beam search do not support `num_return_sequences` different than 1 (got {}).

What it means

Raised by GenerationConfig.validate() when num_return_sequences > 1 but the decoding strategy is greedy search (do_sample is False/None and num_beams is None or 1). Greedy decoding is deterministic: with a single sequence path there is exactly one argmax continuation, so multiple return sequences cannot be produced. The library refuses the configuration instead of silently returning duplicate sequences.

Solutions

1. Set do_sample=True (and typically temperature/top_p) when you want multiple diverse sequences: model.generate(..., do_sample=True, num_return_sequences=5)
2. Or use beam search: set num_beams >= num_return_sequences (e.g. num_beams=5, num_return_sequences=5)
3. Or reduce num_return_sequences to 1 if you only need greedy output
4. If the error comes from a saved generation_config.json, edit that file or override the attributes on model.generation_config before calling generate()

Example fix

# before
out = model.generate(**inputs, num_return_sequences=4)
# after
out = model.generate(**inputs, do_sample=True, temperature=0.7, num_return_sequences=4)
# or beam search
out = model.generate(**inputs, num_beams=4, num_return_sequences=4)

def check_return_sequences(cfg):
    nrs = cfg.num_return_sequences or 1
    beams = cfg.num_beams or 1
    if nrs > 1 and not cfg.do_sample and beams <= 1:
        raise ValueError('num_return_sequences>1 requires do_sample=True or num_beams>=nrs')
    return True

def can_multi_return(cfg) -> bool:
    nrs = cfg.num_return_sequences or 1
    return nrs <= 1 or bool(cfg.do_sample) or (cfg.num_beams or 1) >= nrs

try:
    model.generation_config.validate()
except ValueError as e:
    if 'num_return_sequences' in str(e):
        model.generation_config.do_sample = True  # or set num_beams
    else:
        raise

When it happens

Trigger: Calling model.generate(num_return_sequences=5) without setting do_sample=True or num_beams>=5; or loading a model whose generation_config.json sets num_return_sequences>1 while do_sample stays unset; validate() is invoked on GenerationConfig instantiation, .generate(), or save_pretrained(strict).

Common situations: Porting old sampling code where do_sample used to default to True; copying a generation_config.json from a sampling-tuned checkpoint onto a greedy model; setting num_return_sequences for data augmentation while forgetting sampling.

Related errors

• `early_stopping` must be a boolean or 'never', but is
• Invalid `cache_implementation
• `max_new_tokens` must be greater than 0, but is
• `num_return_sequences
• A GenerationConfig must be provided or set in the model.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/349dbceebd74aecc. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/generation/configuration_utils.py:792

                and _should_warn("num_beams", "early_stopping", user_set_attributes)
            ):
                minor_issues["early_stopping"] = single_beam_wrong_parameter_msg.format(
                    num_beams=self.num_beams, flag_name="early_stopping", flag_value=self.early_stopping
                )
            if (
                self.length_penalty is not None
                and self.length_penalty != 1.0
                and _should_warn("num_beams", "length_penalty", user_set_attributes)
            ):
                minor_issues["length_penalty"] = single_beam_wrong_parameter_msg.format(
                    num_beams=self.num_beams, flag_name="length_penalty", flag_value=self.length_penalty
                )

        # 2.4. check `num_return_sequences`
        if self.num_return_sequences is not None and self.num_return_sequences > 1:
            if self.num_beams is None or self.num_beams == 1:
                if not self.do_sample:
                    raise ValueError(
                        "Greedy methods (do_sample != True) without beam search do not support "
                        f"`num_return_sequences` different than 1 (got {self.num_return_sequences})."
                    )
            elif (
                self.num_beams is not None
                and self.num_return_sequences is not None
                and self.num_return_sequences > self.num_beams
            ):
                raise ValueError(
                    f"`num_return_sequences` ({self.num_return_sequences}) has to be smaller or equal to `num_beams` "
                    f"({self.num_beams})."
                )

        # 2.5. check cache-related arguments
        if self.use_cache is False:
            # In this case, all cache-related arguments should be unset. However, since `use_cache=False` is often used
            # passed to `generate` directly to hot-fix cache issues, let's raise a warning instead of an error
            # (otherwise a user might need to overwrite several parameters).

View on GitHub (pinned to a597f97485)