{"record":{"id":"299bf79a0cf7cac8","repo":"huggingface/transformers","slug":"decoder-start-token-id-or-bos-token-id-has-to","errorCode":null,"errorMessage":"`decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation.","messagePattern":"`decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":2091,"sourceCode":"            )\n\n        # We can have more than one eos token. Always treat it as a 1D tensor (when it exists).\n        if eos_token_tensor is not None and eos_token_tensor.ndim == 0:\n            eos_token_tensor = eos_token_tensor.unsqueeze(0)\n\n        # Set pad token if unset (and there are conditions to do so)\n        if pad_token_tensor is None and eos_token_tensor is not None:\n            # Only emits the warnings if batch_size>1, as batch_size==1 means no padding, thus no problems\n            if kwargs_has_attention_mask is not None and not kwargs_has_attention_mask and is_batched_sequence:\n                logger.warning(\n                    \"The attention mask and the pad token id were not set, with a batched input. As a consequence, you may \"\n                    \"observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\"\n                )\n            pad_token_tensor = eos_token_tensor[0]\n\n        # Sanity checks/warnings\n        if self.config.is_encoder_decoder and decoder_start_token_tensor is None:\n            raise ValueError(\n                \"`decoder_start_token_id` or `bos_token_id` has to be defined for encoder-decoder generation.\"\n            )\n        if eos_token_tensor is not None and torch.isin(eos_token_tensor, pad_token_tensor).any():\n            # Only emits the warning if batch_size>1, as batch_size==1 means no padding, thus no problems\n            if kwargs_has_attention_mask is not None and not kwargs_has_attention_mask and is_batched_sequence:\n                logger.warning_once(\n                    \"The attention mask is not set with a batched input, and cannot be inferred from input because pad token \"\n                    \"is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's \"\n                    \"`attention_mask` to obtain reliable results.\"\n                )\n        if eos_token_tensor is not None and (\n            torch.is_floating_point(eos_token_tensor) or (eos_token_tensor < 0).any()\n        ):\n            logger.warning(\n                f\"`eos_token_id` should consist of positive integers, but is {eos_token_tensor}. Your generation \"\n                \"will not stop until the maximum length is reached. Depending on other flags, it may even crash.\"\n            )\n","sourceCodeStart":2073,"sourceCodeEnd":2109,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L2073-L2109","documentation":"Encoder-decoder generation must know which token starts the decoder. It resolves `decoder_start_token_id` (falling back to `bos_token_id`) from the generation config/model config; if the resulting `decoder_start_token_tensor` is None, `generate` cannot construct `decoder_input_ids` and raises.","triggerScenarios":"`model.generate(**encoder_inputs)` on an encoder-decoder model whose `generation_config.decoder_start_token_id` and `.bos_token_id` are both None — common with custom-trained seq2seq models or hand-built `GenerationConfig`s; calling `generate(input_ids=...)` without `decoder_input_ids`.","commonSituations":"Fine-tuned T5/BART/Whisper-style checkpoints where the start token was implied by trainer code but never stored in the config; programmatic `GenerationConfig(...)` omitting it; converting checkpoints from other frameworks that drop the field.","solutions":["Set it on the generation config: `model.generation_config.decoder_start_token_id = tokenizer.pad_token_id` (or the correct start id, e.g. `tokenizer.bos_token_id` / `tokenizer.convert_tokens_to_ids(tokenizer.lang_code_to_token[\"en\"])` for mBART-style models).","Or pass it per call: `model.generate(**enc, decoder_start_token_id=start_id)`.","Or supply `decoder_input_ids` explicitly so the start token is not needed.","Persist the fix: update the checkpoint's `generation_config.json` so reloads work."],"exampleFix":"# before\nout = model.generate(input_ids=encoder_input_ids)  # decoder start undefined -> ValueError\n\n# after\nmodel.generation_config.decoder_start_token_id = tokenizer.bos_token_id\nout = model.generate(input_ids=encoder_input_ids)","handlingStrategy":"validation","validationCode":"if model.config.is_encoder_decoder:\n    gc = model.generation_config\n    if gc.decoder_start_token_id is None and gc.bos_token_id is None:\n        gc.decoder_start_token_id = tokenizer.bos_token_id if tokenizer.bos_token_id is not None else tokenizer.pad_token_id","typeGuard":null,"tryCatchPattern":null,"preventionTips":["For seq2seq checkpoints you train/export yourself, always set decoder_start_token_id in generation_config.json.","Smoke-test generate() after loading a fine-tuned encoder-decoder model.","Know your model's start token convention (T5 pad, mBART language code) and pin it explicitly."],"tags":["generation","encoder-decoder","decoder-start-token","config"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}