Lightning-AI/pytorch-lightning · warning

The PyTorch Profiler default schedule will be overridden as

Error message

The PyTorch Profiler default schedule will be overridden as there is not enough steps to properly record traces.

What it means

PyTorch Profiler's default schedule (wait=1, warmup=1, active=3) needs at least 5 steps; with fewer steps a segmentation fault would occur, so Lightning overrides the schedule to none and warns.

Source

Thrown at src/lightning/pytorch/profilers/pytorch.py:460

    @override
    def stop(self, action_name: str) -> None:
        if action_name in self._recording_map:
            self._recording_map[action_name].__exit__(None, None, None)
            del self._recording_map[action_name]

        if not _KINETO_AVAILABLE or self._emit_nvtx:
            return

        if self.profiler is not None and any(action_name.endswith(func) for func in self.STEP_FUNCTIONS):
            assert isinstance(self.profiler, torch.profiler.profile)
            if self._schedule is not None:
                self._schedule.pre_step(action_name)

            # the default schedule requires a minimum of 5 steps to properly work: `wait=1, warmup=1, active=3`.
            # otherwise, this will raise a `segmentation fault`.
            if self._should_override_schedule():
                warning_cache.warn(
                    "The PyTorch Profiler default schedule will be overridden as there is not enough "
                    "steps to properly record traces."
                )
                self._schedule = None
                self.profiler.schedule = torch.profiler.profiler._default_schedule_fn

            def on_trace_ready(profiler: _PROFILER) -> None:
                if self.dirpath is not None:
                    if self._export_to_chrome:
                        handler = tensorboard_trace_handler(
                            str(self.dirpath), self._prepare_filename(action_name=action_name, extension="")
                        )
                        handler(profiler)

                    if self._export_to_flame_graph:
                        path = os.path.join(
                            self.dirpath, self._prepare_filename(action_name=action_name, extension=".stack")
                        )

View on GitHub (pinned to 9fed5c27d2)

Solutions

  1. Increase steps: run with at least 5 optimizer steps (more data or epochs)
  2. Or set an explicit schedule with fewer steps: schedule=torch.profiler.schedule(wait=0, warmup=0, active=1) via ProfilerAction config
  3. Or accept the override — profiling still works, just unscheduled

Example fix

# before
profiler = PyTorchProfiler()  # default schedule, <5 steps
trainer = Trainer(profiler=profiler, fast_dev_run=True)
# after
profiler = PyTorchProfiler(schedule=torch.profiler.schedule(wait=0, warmup=0, active=3))
trainer = Trainer(profiler=profiler, limit_train_batches=4)
Defensive patterns

Strategy: validation

Validate before calling

steps = min(trainer.max_steps or 10**9, num_batches)
assert steps >= 5 or custom_schedule_set, 'profiler schedule needs >=5 steps'

Prevention

When it happens

Trigger: Trainer(profiler=PyTorchProfiler(...)) with default schedule and fast_dev_run or max_steps/train batches < 5, or running predict/validate with very few batches.

Common situations: Quick profiling smoke tests with tiny datasets; fast_dev_run=True combined with the profiler.

Related errors


AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/e8b6df24770585ca. Report an issue: GitHub.