{"record":{"id":"04bd285b3e992c60","repo":"huggingface/transformers","slug":"bos-token-id-has-to-be-defined-when-no-input-id","errorCode":null,"errorMessage":"`bos_token_id` has to be defined when no `input_ids` are provided.","messagePattern":"`bos_token_id` has to be defined when no `input_ids` are provided\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/generation/utils.py","lineNumber":747,"sourceCode":"        # soft-prompting or in multimodal implementations built on top of decoder-only language models.\n        batch_size = 1\n        for value in model_kwargs.values():\n            if isinstance(value, torch.Tensor):\n                batch_size = value.shape[0]\n                break\n\n        if \"inputs_embeds\" in model_kwargs:\n            return torch.ones(\n                (batch_size, 0),\n                dtype=torch.long,\n                # Use the device of the existing tensor to avoid any potential `meta` device issue, which is likely\n                # linked to the offloading behavior (keeping it on meta device). See PR #44848. Previously, it used\n                # `self.device`.\n                device=self.device if self.device.type != \"meta\" else model_kwargs[\"inputs_embeds\"].device,\n            )\n\n        if bos_token_id is None:\n            raise ValueError(\"`bos_token_id` has to be defined when no `input_ids` are provided.\")\n\n        return torch.ones((batch_size, 1), dtype=torch.long, device=self.device) * bos_token_id\n\n    def _prepare_position_ids_for_generation(self, inputs_tensor, model_kwargs):\n        \"\"\"\n        Tries to infer position ids given attention mask and past kv cache length. All instances when\n        `position_ids=None` should call this method.\n        \"\"\"\n        # `input_ids` may be present in the model kwargs, instead of being the main input (e.g. multimodal model)\n        if \"input_ids\" in model_kwargs and model_kwargs[\"input_ids\"].shape[1] > 0:\n            inputs_tensor = model_kwargs[\"input_ids\"]\n\n        seq_length = inputs_tensor.shape[1]\n\n        if (attention_mask := model_kwargs.get(\"attention_mask\")) is not None:\n            position_ids = attention_mask.long().cumsum(-1) - 1\n            # We need this as otherwise padding tokens appear as -1 in position\n            position_ids = position_ids.masked_fill(attention_mask == 0, 0)","sourceCodeStart":729,"sourceCodeEnd":765,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/generation/utils.py#L729-L765","documentation":"ValueError from _maybe_initialize_input_ids_for_generation: no input_ids were given (and no usable inputs_embeds), so generate wants to bootstrap a (batch_size, 1) tensor filled with bos_token_id — but the config/generation_config has bos_token_id=None. Without either a prompt or a BOS id there is nothing valid to start decoding from.","triggerScenarios":"model.generate(batch_size=2, max_new_tokens=50) (prompt-free generation) on a model whose generation_config.bos_token_id is None; models like Whisper or some Gemma configs where bos is not set; passing only attention_mask/model_kwargs without ids.","commonSituations":"Unconditional sampling scripts; models with stripped generation configs after conversion/quantization; assuming generate() can start from a learned start token that the config does not declare.","solutions":["Pass input_ids explicitly, e.g. input_ids=torch.full((1,1), tokenizer.bos_token_id, ...).","Or set model.generation_config.bos_token_id = tokenizer.bos_token_id before calling generate.","If you meant conditioned generation, make sure the prompt tensor is actually forwarded (not dropped by a wrapper)."],"exampleFix":"# before\nout = model.generate(batch_size=1, max_new_tokens=32)  # bos_token_id is None\n\n# after\nout = model.generate(input_ids=torch.tensor([[tokenizer.bos_token_id]]), max_new_tokens=32)","handlingStrategy":"validation","validationCode":"bos = getattr(model.generation_config, \"bos_token_id\", None)\nif input_ids is None and bos is None:\n    input_ids = torch.full((batch_size, 1), tokenizer.bos_token_id, dtype=torch.long, device=model.device)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["For unconditional generation always pass an explicit BOS tensor or set generation_config.bos_token_id first.","Validate after loading a checkpoint that generation_config has bos_token_id when you plan prompt-free sampling."],"tags":["generate","bos-token","config","unconditional-generation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}