PaddlePaddle/PaddleOCR · error · ValueError

Tried to step {} times. The specified number of total steps

Error message

Tried to step {} times. The specified number of total steps is {}

What it means

Raised from OneCycleDecay.get_lr() once last_epoch exceeds the total_steps the scheduler was built with. OneCycleDecay precomputes a fixed phase schedule (warmup then decay) over total_steps, so stepping past that budget has no defined LR and is rejected at runtime, typically deep inside a training loop.

Source

Thrown at ppocr/optimizer/lr_scheduler.py:151

            self.anneal_func = self._annealing_linear

        super(OneCycleDecay, self).__init__(max_lr, last_epoch, verbose)

    def _annealing_cos(self, start, end, pct):
        "Cosine anneal from `start` to `end` as pct goes from 0.0 to 1.0."
        cos_out = math.cos(math.pi * pct) + 1
        return end + (start - end) / 2.0 * cos_out

    def _annealing_linear(self, start, end, pct):
        "Linearly anneal from `start` to `end` as pct goes from 0.0 to 1.0."
        return (end - start) * pct + start

    def get_lr(self):
        computed_lr = 0.0
        step_num = self.last_epoch

        if step_num > self.total_steps:
            raise ValueError(
                "Tried to step {} times. The specified number of total steps is {}".format(
                    step_num + 1, self.total_steps
                )
            )
        start_step = 0
        for i, phase in enumerate(self._schedule_phases):
            end_step = phase["end_step"]
            if step_num <= end_step or i == len(self._schedule_phases) - 1:
                pct = (step_num - start_step) / (end_step - start_step)
                computed_lr = self.anneal_func(phase["start_lr"], phase["end_lr"], pct)
                break
            start_step = phase["end_step"]

        return computed_lr


class TwoStepCosineDecay(LRScheduler):
    def __init__(

View on GitHub (pinned to 2661c7c0ef)

Solutions

  1. Recompute total_steps from the actual dataloader: total_steps = epochs * len(train_dataloader) per process, and rebuild the scheduler.
  2. If resuming/extending training, rebuild OneCycleDecay with the new larger total_steps rather than reusing the pickled scheduler.
  3. Verify you call scheduler.step() once per optimizer step, not per batch sub-iteration or per log interval.

Example fix

# before
total_steps = epochs * steps_per_epoch  # stale estimate after batch size change

# after
steps_per_epoch = math.ceil(n_samples / (batch_size * world_size))
total_steps = epochs * steps_per_epoch
OneCycleDecay(max_lr=0.001, total_steps=total_steps, pct_start=0.1)
Defensive patterns

Strategy: validation

Validate before calling

planned_steps = epochs * math.ceil(n_samples / (batch_size * world_size))
scheduler = OneCycleDecay(max_lr=lr, total_steps=planned_steps, pct_start=0.1)
# guard inside a thin training-loop wrapper:
if scheduler.last_epoch >= scheduler.total_steps:
    raise RuntimeError(f"total_steps={scheduler.total_steps} exhausted; rebuild scheduler with the new epoch budget")

Try / catch

try:
    lr = scheduler.get_lr()
except ValueError:
    # budget exhausted: rebuild the schedule for the new horizon and continue
    scheduler = rebuild_one_cycle(total_steps=new_total_steps)
    lr = scheduler.get_lr()

Prevention

When it happens

Trigger: The scheduler's total_steps is smaller than the actual number of optimizer steps taken: epochs/steps_per_epoch miscounted in config, resuming a checkpoint and continuing to train past the planned budget, or stepping the LR scheduler more often than the optimizer.

Common situations: Config computes total_steps = epochs * steps_per_epoch but the dataloader length or world size changed (more GPUs, smaller batch), so real steps exceed the estimate; fine-tuning jobs extended beyond the original epoch count without regenerating total_steps.

Related errors

AI-assisted analysis of PaddlePaddle/PaddleOCR@2661c7c0ef (2026-08-14). Data as JSON: /api/errors/6dfdc272bc46518a. Report an issue: GitHub.