huggingface/transformers · error · ValueError

You passed `inputs_embeds` and `input_ids` to `.generate()`.

Error message

You passed `inputs_embeds` and `input_ids` to `.generate()`. Please pick one.

What it means

ValueError for encoder-decoder models: both inputs_embeds and input_ids were supplied to .generate(). For encoder-decoder models inputs_embeds replaces the encoder inputs (input_ids), so passing both is ambiguous and generate asks you to pick exactly one.

Source

Thrown at src/transformers/generation/utils.py:704

            elif not self.config.is_encoder_decoder:
                has_inputs_embeds_forwarding = "inputs_embeds" in set(
                    inspect.signature(self.prepare_inputs_for_generation).parameters.keys()
                )
                if not has_inputs_embeds_forwarding:
                    raise ValueError(
                        f"You passed `inputs_embeds` to `.generate()`, but the model class {self.__class__.__name__} "
                        "doesn't have its forwarding implemented. See the GPT2 implementation for an example "
                        "(https://github.com/huggingface/transformers/pull/21405), and feel free to open a PR with it!"
                    )
                # In this case, `input_ids` is moved to the `model_kwargs`, so a few automations (like the creation of
                # the attention mask) can rely on the actual model input.
                model_kwargs["input_ids"] = self._maybe_initialize_input_ids_for_generation(
                    inputs, bos_token_id, model_kwargs=model_kwargs
                )
                inputs, input_name = model_kwargs["inputs_embeds"], "inputs_embeds"
            else:
                if inputs is not None:
                    raise ValueError("You passed `inputs_embeds` and `input_ids` to `.generate()`. Please pick one.")
                inputs, input_name = model_kwargs["inputs_embeds"], "inputs_embeds"

        # 4. if `inputs` is still None, try to create `input_ids` from BOS token
        inputs = self._maybe_initialize_input_ids_for_generation(inputs, bos_token_id, model_kwargs)
        return inputs, input_name, model_kwargs

    def _maybe_initialize_input_ids_for_generation(
        self: "GenerativePreTrainedModel",
        inputs: torch.Tensor | None,
        bos_token_id: torch.Tensor | None,
        model_kwargs: dict[str, torch.Tensor],
    ) -> torch.LongTensor:
        """Initializes input ids for generation, if necessary."""
        if inputs is not None:
            return inputs

        encoder_outputs = model_kwargs.get("encoder_outputs")
        last_hidden_state = getattr(encoder_outputs, "last_hidden_state", None)

View on GitHub (pinned to a597f97485)

Solutions

Remove one of the two: pass inputs_embeds alone (it becomes the encoder input) or input_ids alone.
Sanitize kwargs: if 'inputs_embeds' in kwargs: kwargs.pop('input_ids', None) before generate.
Check that a default-generation wrapper is not auto-inserting input_ids.

Example fix

# before
out = model.generate(input_ids=ids, inputs_embeds=embeds, max_new_tokens=10)

# after
out = model.generate(inputs_embeds=embeds, max_new_tokens=10)

Defensive patterns

Strategy: validation

Validate before calling

if model.config.is_encoder_decoder and "inputs_embeds" in kwargs:
    kwargs.pop("input_ids", None)  # embeds replaces encoder input_ids

Prevention

Enforce mutual exclusion of input_ids and inputs_embeds in your generate wrapper.
Log which encoder input route is active before calling generate.

When it happens

Trigger: model.generate(input_ids=enc_ids, inputs_embeds=enc_embeds, decoder_start_token_id=...) on an encoder-decoder model; helper code that always sets input_ids and then conditionally adds inputs_embeds.

Common situations: Soft-prompt/continuous-encoder experiments on T5/BART-style models; merging default kwargs with user kwargs producing both keys.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/f56e94f9d48ae5bf. Report an issue: GitHub.