{"record":{"id":"c20e4a06f72aa0f5","repo":"huggingface/transformers","slug":"you-passed-inputs-embeds-to-generate-but-t","errorCode":null,"errorMessage":"You passed `inputs_embeds` to `.generate()`, but the model class {self.__class__.__name__} doesn't have its forwarding implemented. See the GPT2 implementation for an example (https://github.com/huggingface/transformers/pull/21405), and feel free to open a PR with it!","messagePattern":"You passed `inputs_embeds` to `\\.generate\\(\\)`, but the model class (.+?) doesn't have its forwarding implemented\\. See the GPT2 implementation for an example \\(https://github\\.com/huggingface/transformers/pull/21405\\), and feel free to open a PR with it!","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":691,"sourceCode":"            )\n        elif inputs_kwarg is not None:\n            inputs = inputs_kwarg\n\n        # 3. In the presence of `inputs_embeds` for text models:\n        # - decoder-only models should complain if the user attempts to pass `inputs_embeds`, but the model\n        # doesn't have its forwarding implemented. `inputs_embeds` is kept in `model_kwargs` and can coexist with\n        # input_ids (`inputs_embeds` will be used in the 1st generation step, as opposed to `input_ids`)\n        # - encoder-decoder models should complain if the user attempts to pass `inputs_embeds` and `input_ids`, and\n        # pull the former to inputs. It will be used in place of `input_ids` to get the encoder hidden states.\n        if input_name == \"input_ids\" and \"inputs_embeds\" in model_kwargs:\n            if model_kwargs[\"inputs_embeds\"] is None:\n                model_kwargs.pop(\"inputs_embeds\")\n            elif not self.config.is_encoder_decoder:\n                has_inputs_embeds_forwarding = \"inputs_embeds\" in set(\n                    inspect.signature(self.prepare_inputs_for_generation).parameters.keys()\n                )\n                if not has_inputs_embeds_forwarding:\n                    raise ValueError(\n                        f\"You passed `inputs_embeds` to `.generate()`, but the model class {self.__class__.__name__} \"\n                        \"doesn't have its forwarding implemented. See the GPT2 implementation for an example \"\n                        \"(https://github.com/huggingface/transformers/pull/21405), and feel free to open a PR with it!\"\n                    )\n                # In this case, `input_ids` is moved to the `model_kwargs`, so a few automations (like the creation of\n                # the attention mask) can rely on the actual model input.\n                model_kwargs[\"input_ids\"] = self._maybe_initialize_input_ids_for_generation(\n                    inputs, bos_token_id, model_kwargs=model_kwargs\n                )\n                inputs, input_name = model_kwargs[\"inputs_embeds\"], \"inputs_embeds\"\n            else:\n                if inputs is not None:\n                    raise ValueError(\"You passed `inputs_embeds` and `input_ids` to `.generate()`. Please pick one.\")\n                inputs, input_name = model_kwargs[\"inputs_embeds\"], \"inputs_embeds\"\n\n        # 4. if `inputs` is still None, try to create `input_ids` from BOS token\n        inputs = self._maybe_initialize_input_ids_for_generation(inputs, bos_token_id, model_kwargs)\n        return inputs, input_name, model_kwargs","sourceCodeStart":673,"sourceCodeEnd":709,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L673-L709","documentation":"ValueError during input preparation: you passed inputs_embeds to .generate() on a decoder-only model whose prepare_inputs_for_generation does not accept an inputs_embeds parameter. The generation loop needs the model to be able to consume embeddings at least for the first forward step; support is opt-in per model architecture (see GPT2's implementation referenced in the message).","triggerScenarios":"model.generate(inputs_embeds=embeds, ...) with a model class whose prepare_inputs_for_generation signature lacks inputs_embeds; older custom models or architectures that never implemented embedding-level generation.","commonSituations":"Prefix-continuation pipelines that precompute hidden states; embedding-based steering/soft-prompt tooling applied to a model without embeds support; upgrading transformers where a custom model's overridden prepare_inputs_for_generation dropped the parameter.","solutions":["Switch to a model with inputs_embeds support in generation (GPT2 family, most modern decoder-only models).","If it is your custom model, add inputs_embeds to prepare_inputs_for_generation and forward it through (copy the GPT2 pattern from the linked PR #21405).","As a workaround, map embeddings back to tokens or append via input_ids instead."],"exampleFix":"# before\nout = model.generate(inputs_embeds=soft_prompt_embeds, max_new_tokens=20)  # raises\n\n# after  (custom model)\nclass MyModel(PreTrainedModel):\n    def prepare_inputs_for_generation(self, input_ids, past_key_values=None, inputs_embeds=None, **kwargs):\n        ...  # accept and forward inputs_embeds like GPT2 does","handlingStrategy":"validation","validationCode":"import inspect\n\nsupports_embeds = \"inputs_embeds\" in inspect.signature(\n    model.prepare_inputs_for_generation\n).parameters\nif not supports_embeds:\n    raise ValueError(f\"{type(model).__name__} cannot generate from inputs_embeds\")","typeGuard":"def model_supports_embeds_generation(model) -> bool:\n    return \"inputs_embeds\" in inspect.signature(model.prepare_inputs_for_generation).parameters","tryCatchPattern":null,"preventionTips":["Feature-detect prepare_inputs_for_generation before building embeds-based pipelines.","Keep custom models' prepare_inputs_for_generation signatures in sync with upstream (include inputs_embeds).","Pin model architectures known to support embeds (GPT2-style) for soft-prompt tooling."],"tags":["generate","inputs-embeds","model-architecture","custom-model"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}