{"record":{"id":"6dfdc272bc46518a","repo":"PaddlePaddle/PaddleOCR","slug":"tried-to-step-times-the-specified-number-of-to","errorCode":null,"errorMessage":"Tried to step {} times. The specified number of total steps is {}","messagePattern":"Tried to step (.+?) times\\. The specified number of total steps is (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ppocr/optimizer/lr_scheduler.py","lineNumber":151,"sourceCode":"            self.anneal_func = self._annealing_linear\n\n        super(OneCycleDecay, self).__init__(max_lr, last_epoch, verbose)\n\n    def _annealing_cos(self, start, end, pct):\n        \"Cosine anneal from `start` to `end` as pct goes from 0.0 to 1.0.\"\n        cos_out = math.cos(math.pi * pct) + 1\n        return end + (start - end) / 2.0 * cos_out\n\n    def _annealing_linear(self, start, end, pct):\n        \"Linearly anneal from `start` to `end` as pct goes from 0.0 to 1.0.\"\n        return (end - start) * pct + start\n\n    def get_lr(self):\n        computed_lr = 0.0\n        step_num = self.last_epoch\n\n        if step_num > self.total_steps:\n            raise ValueError(\n                \"Tried to step {} times. The specified number of total steps is {}\".format(\n                    step_num + 1, self.total_steps\n                )\n            )\n        start_step = 0\n        for i, phase in enumerate(self._schedule_phases):\n            end_step = phase[\"end_step\"]\n            if step_num <= end_step or i == len(self._schedule_phases) - 1:\n                pct = (step_num - start_step) / (end_step - start_step)\n                computed_lr = self.anneal_func(phase[\"start_lr\"], phase[\"end_lr\"], pct)\n                break\n            start_step = phase[\"end_step\"]\n\n        return computed_lr\n\n\nclass TwoStepCosineDecay(LRScheduler):\n    def __init__(","sourceCodeStart":133,"sourceCodeEnd":169,"githubUrl":"https://github.com/PaddlePaddle/PaddleOCR/blob/2661c7c0ef5c613e8f93c6e93b2e052399f0f854/ppocr/optimizer/lr_scheduler.py#L133-L169","documentation":"Raised from OneCycleDecay.get_lr() once last_epoch exceeds the total_steps the scheduler was built with. OneCycleDecay precomputes a fixed phase schedule (warmup then decay) over total_steps, so stepping past that budget has no defined LR and is rejected at runtime, typically deep inside a training loop.","triggerScenarios":"The scheduler's total_steps is smaller than the actual number of optimizer steps taken: epochs/steps_per_epoch miscounted in config, resuming a checkpoint and continuing to train past the planned budget, or stepping the LR scheduler more often than the optimizer.","commonSituations":"Config computes total_steps = epochs * steps_per_epoch but the dataloader length or world size changed (more GPUs, smaller batch), so real steps exceed the estimate; fine-tuning jobs extended beyond the original epoch count without regenerating total_steps.","solutions":["Recompute total_steps from the actual dataloader: total_steps = epochs * len(train_dataloader) per process, and rebuild the scheduler.","If resuming/extending training, rebuild OneCycleDecay with the new larger total_steps rather than reusing the pickled scheduler.","Verify you call scheduler.step() once per optimizer step, not per batch sub-iteration or per log interval."],"exampleFix":"# before\ntotal_steps = epochs * steps_per_epoch  # stale estimate after batch size change\n\n# after\nsteps_per_epoch = math.ceil(n_samples / (batch_size * world_size))\ntotal_steps = epochs * steps_per_epoch\nOneCycleDecay(max_lr=0.001, total_steps=total_steps, pct_start=0.1)","handlingStrategy":"validation","validationCode":"planned_steps = epochs * math.ceil(n_samples / (batch_size * world_size))\nscheduler = OneCycleDecay(max_lr=lr, total_steps=planned_steps, pct_start=0.1)\n# guard inside a thin training-loop wrapper:\nif scheduler.last_epoch >= scheduler.total_steps:\n    raise RuntimeError(f\"total_steps={scheduler.total_steps} exhausted; rebuild scheduler with the new epoch budget\")","typeGuard":null,"tryCatchPattern":"try:\n    lr = scheduler.get_lr()\nexcept ValueError:\n    # budget exhausted: rebuild the schedule for the new horizon and continue\n    scheduler = rebuild_one_cycle(total_steps=new_total_steps)\n    lr = scheduler.get_lr()","preventionTips":["Derive total_steps from len(train_dataloader) at runtime instead of hardcoding.","When extending or resuming training, construct a fresh scheduler with the updated total_steps.","Step the scheduler exactly once per optimizer step."],"tags":["lr-scheduler","training","config","runtime"],"backgroundTag":null,"analyzedSha":"2661c7c0ef5c613e8f93c6e93b2e052399f0f854","analyzedAt":"2026-08-14T20:17:30.180Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}