{"record":{"id":"26e843e37643fe81","repo":"PaddlePaddle/PaddleOCR","slug":"if-eos-token-id-is-defined-make-sure-that-pad-26e843","errorCode":null,"errorMessage":"If `eos_token_id` is defined, make sure that `pad_token_id` is defined.","messagePattern":"If `eos_token_id` is defined, make sure that `pad_token_id` is defined\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ppocr/modeling/heads/rec_unimernet_head.py","lineNumber":2464,"sourceCode":"        pad_token_id = self.pad_token_id\n        eos_token_id = [self.eos_token_id]\n        eos_token = self.eos_token_id\n        unfinished_sequences = paddle.ones(batch_size, dtype=paddle.int64)\n        for idx in range(self.max_seq_len):\n            model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)\n            outputs = self.generate_single_iter(\n                **model_inputs,\n                return_dict=True,\n                output_attentions=False,\n                output_hidden_states=False,\n            )\n            next_token_logits = outputs.logits[:, -1, :]\n\n            next_tokens_scores = self.logits_processor(input_ids, next_token_logits)\n            next_tokens = paddle.argmax(next_tokens_scores, axis=-1)\n            if eos_token_id is not None:\n                if pad_token_id is None:\n                    raise ValueError(\n                        \"If `eos_token_id` is defined, make sure that `pad_token_id` is defined.\"\n                    )\n                next_tokens = next_tokens * unfinished_sequences + pad_token_id * (\n                    1 - unfinished_sequences\n                )\n            input_ids = paddle.concat([input_ids, next_tokens[:, None]], axis=-1)\n            model_kwargs = self._update_model_kwargs_for_generation(\n                outputs,\n                model_kwargs,\n                is_encoder_decoder=self.config_decoder.is_encoder_decoder,\n            )\n            unfinished_sequences = unfinished_sequences & ~self.stopping_criteria(\n                input_ids\n            ).cast(paddle.int64)\n\n            if (\n                eos_token is not None\n                and (","sourceCodeStart":2446,"sourceCodeEnd":2482,"githubUrl":"https://github.com/PaddlePaddle/PaddleOCR/blob/2661c7c0ef5c613e8f93c6e93b2e052399f0f854/ppocr/modeling/heads/rec_unimernet_head.py#L2446-L2482","documentation":"In the manual greedy generation loop, once sequences finish they are replaced by pad_token_id (next_tokens * unfinished + pad * finished). If eos_token_id is configured but pad_token_id is not, the loop cannot mark finished sequences, so it raises this ValueError demanding pad_token_id be defined.","triggerScenarios":"Running generate on the UniMERNet head with generation_config (or generate args) containing eos_token_id but pad_token_id=None.","commonSituations":"Configs ported from HF models that relied on the library's default pad fallback; user tokenizers where pad was never assigned; fine-tuned checkpoints that saved partial generation configs.","solutions":["Set pad_token_id in generation_config (commonly equal to eos or a dedicated pad token)","Or pass pad_token_id=... explicitly to generate()","If no pad token exists, map pad_token_id to eos_token_id"],"exampleFix":"# before\nGenerationConfig(eos_token_id=2)\n# after\nGenerationConfig(eos_token_id=2, pad_token_id=2)","handlingStrategy":"validation","validationCode":"if generation_config.eos_token_id is not None:\n    generation_config.pad_token_id = generation_config.pad_token_id or generation_config.eos_token_id","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always define pad_token_id alongside eos_token_id in generation configs","Map pad to eos when the tokenizer has no dedicated pad token"],"tags":["paddle","generation","config","token-ids"],"backgroundTag":null,"analyzedSha":"2661c7c0ef5c613e8f93c6e93b2e052399f0f854","analyzedAt":"2026-08-14T20:17:30.180Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}