huggingface/transformers · error · ValueError

`num_return_sequences` ({}) has to be smaller or equal to `n

Error message

`num_return_sequences` ({}) has to be smaller or equal to `num_beams` ({}).

What it means

Raised by GenerationConfig.validate() when beam search is active (num_beams > 1) but num_return_sequences exceeds num_beams. Each return sequence is drawn from the beam set, so at most num_beams distinct sequences can be returned; requesting more is rejected as a configuration error.

Source

Thrown at src/transformers/generation/configuration_utils.py:801

            ):
                minor_issues["length_penalty"] = single_beam_wrong_parameter_msg.format(
                    num_beams=self.num_beams, flag_name="length_penalty", flag_value=self.length_penalty
                )

        # 2.4. check `num_return_sequences`
        if self.num_return_sequences is not None and self.num_return_sequences > 1:
            if self.num_beams is None or self.num_beams == 1:
                if not self.do_sample:
                    raise ValueError(
                        "Greedy methods (do_sample != True) without beam search do not support "
                        f"`num_return_sequences` different than 1 (got {self.num_return_sequences})."
                    )
            elif (
                self.num_beams is not None
                and self.num_return_sequences is not None
                and self.num_return_sequences > self.num_beams
            ):
                raise ValueError(
                    f"`num_return_sequences` ({self.num_return_sequences}) has to be smaller or equal to `num_beams` "
                    f"({self.num_beams})."
                )

        # 2.5. check cache-related arguments
        if self.use_cache is False:
            # In this case, all cache-related arguments should be unset. However, since `use_cache=False` is often used
            # passed to `generate` directly to hot-fix cache issues, let's raise a warning instead of an error
            # (otherwise a user might need to overwrite several parameters).
            no_cache_warning = (
                "You have not set `use_cache` to `True`, but {cache_arg} is set to {cache_arg_value}."
                "{cache_arg} will have no effect."
            )
            for arg_name in ("cache_implementation", "cache_config"):
                if getattr(self, arg_name) is not None:
                    minor_issues[arg_name] = no_cache_warning.format(
                        cache_arg=arg_name, cache_arg_value=getattr(self, arg_name)
                    )

View on GitHub (pinned to a597f97485)

Solutions

1. Raise num_beams to at least num_return_sequences: model.generate(num_beams=5, num_return_sequences=5)
2. Or lower num_return_sequences to <= num_beams
3. Or switch to sampling (do_sample=True) where num_return_sequences is unconstrained by beams
4. Fix the offending values in the model's generation_config.json if they come from there

Example fix

# before
out = model.generate(**inputs, num_beams=2, num_return_sequences=4)
# after
out = model.generate(**inputs, num_beams=4, num_return_sequences=4)

beams = cfg.num_beams or 1
nrs = cfg.num_return_sequences or 1
if beams > 1 and nrs > beams:
    cfg.num_beams = nrs  # or clamp nrs

def beams_ok(cfg) -> bool:
    nrs = cfg.num_return_sequences or 1
    return nrs <= 1 or (cfg.num_beams or 1) >= nrs or bool(cfg.do_sample)

try:
    out = model.generate(**inputs, num_beams=2, num_return_sequences=4)
except ValueError as e:
    if 'num_return_sequences' in str(e) and 'num_beams' in str(e):
        out = model.generate(**inputs, num_beams=4, num_return_sequences=4)
    else:
        raise

When it happens

Trigger: model.generate(num_beams=3, num_return_sequences=5) with do_sample False; a generation_config.json containing e.g. num_beams=2 and num_return_sequences=4; validate() during config load or save_pretrained.

Common situations: Tuning num_return_sequences up for batch generation while leaving num_beams from an earlier experiment; inheriting a beam-search config and adding sequence returns on top.

Related errors

• `early_stopping` must be a boolean or 'never', but is {}.
• `max_new_tokens` must be greater than 0, but is {}.
• Invalid `cache_implementation` ({}). Choose one of: {}
• Greedy methods (do_sample != True) without beam search do no
• Per-component `config` dict is missing entries for: {sorted(

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/fefcbf7090753fdb. Report an issue: GitHub.