{"record":{"id":"936696e9be82e97b","repo":"huggingface/transformers","slug":"if-is-encoder-decoder-is-true-make-sure-that-e","errorCode":null,"errorMessage":"If `is_encoder_decoder` is True, make sure that `encoder_outputs` is defined.","messagePattern":"If `is_encoder_decoder` is True, make sure that `encoder_outputs` is defined\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":935,"sourceCode":"        # Do not call torch.repeat_interleave if expand_size is 1 because it clones\n        # the input tensor and thus requires more memory although no change is applied\n        if expand_size == 1:\n            return input_ids, model_kwargs\n\n        def _expand_dict_for_generation(dict_to_expand):\n            for key in dict_to_expand:\n                if dict_to_expand[key] is not None and isinstance(dict_to_expand[key], torch.Tensor):\n                    dict_to_expand[key] = dict_to_expand[key].repeat_interleave(expand_size, dim=0)\n            return dict_to_expand\n\n        if input_ids is not None:\n            input_ids = input_ids.repeat_interleave(expand_size, dim=0)\n\n        model_kwargs = _expand_dict_for_generation(model_kwargs)\n\n        if is_encoder_decoder:\n            if model_kwargs.get(\"encoder_outputs\") is None:\n                raise ValueError(\"If `is_encoder_decoder` is True, make sure that `encoder_outputs` is defined.\")\n            model_kwargs[\"encoder_outputs\"] = _expand_dict_for_generation(model_kwargs[\"encoder_outputs\"])\n\n        return input_ids, model_kwargs\n\n    def _update_model_kwargs_for_generation(\n        self,\n        outputs: ModelOutput,\n        model_kwargs: dict[str, Any],\n        is_encoder_decoder: bool = False,\n        num_new_tokens: int = 1,\n    ) -> dict[str, Any]:\n        \"\"\"\n        Update the model kwargs to account for the `num_new_tokens` new tokens that were just generated.\n        That is, update the `attention_mask`, `position_ids`, and `token_type_ids` to account for the\n        new tokens of the total sequence.\n        Note that this function never slices inputs, this is performed in `prepare_inputs_for_generation`.\n        \"\"\"\n        # update past_key_values keeping its naming used in model code","sourceCodeStart":917,"sourceCodeEnd":953,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L917-L953","documentation":"ValueError from _expand_inputs_for_generation: for encoder-decoder models, generation with num_return_sequences > 1 or beam search expands inputs and model_kwargs by a factor; the encoder_outputs must already be present in model_kwargs at that point. If it is None/missing, decoding has nothing to condition on, so the expansion helper refuses.","triggerScenarios":"Calling low-level generation utilities (or a custom loop) with is_encoder_decoder=True but no encoder_outputs in model_kwargs — e.g. calling model.generate() on an encoder-decoder model after popping encoder_outputs, or using decoder_start_token_ids without running the encoder first.","commonSituations":"Custom beam-search reimplementations that reuse _expand_inputs_for_generation; caches cleared between encoder and decoder phases; multi-modal encoder-decoder setups where the encoder step was skipped.","solutions":["Run the encoder first and put its output in model_kwargs: model_kwargs['encoder_outputs'] = model.get_encoder()(...), then call generate.","Or simply call model.generate(input_ids=...) on the full encoder-decoder model so the encoder runs internally.","If you intentionally precompute encoder outputs, make sure they survive in model_kwargs until expansion."],"exampleFix":"# before\nmodel._expand_inputs_for_generation(input_ids, model_kwargs, expand_size=4, is_encoder_decoder=True)\n# model_kwargs lacks encoder_outputs\n\n# after\nmodel_kwargs['encoder_outputs'] = model.get_encoder()(encoder_input_ids)\ninput_ids, model_kwargs = model._expand_inputs_for_generation(input_ids, model_kwargs, expand_size=4, is_encoder_decoder=True)","handlingStrategy":"validation","validationCode":"if is_encoder_decoder and model_kwargs.get(\"encoder_outputs\") is None:\n    model_kwargs[\"encoder_outputs\"] = model.get_encoder()(input_ids=encoder_ids)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Prefer the public model.generate(...) which runs the encoder for you.","In custom loops, run and store encoder_outputs before any input expansion step.","Don't pop encoder_outputs from model_kwargs between phases."],"tags":["generate","encoder-decoder","beam-search","internal-api"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}