Lightning-AI/pytorch-lightning · error · ValueError

`num_processes` should be >= 1, got {num_processes}.

Error message

`num_processes` should be >= 1, got {num_processes}.

What it means

Raised by Lightning's `_suggested_max_num_threads` when the number of processes used to derive a per-process CPU thread count is less than 1. The helper divides available CPUs by `num_processes` to suggest an `OMP_NUM_THREADS` value, so a non-positive process count is meaningless. It is typically reached via `_set_num_threads_if_needed` when Lightning or torchrun sets thread counts at launch.

Source

Thrown at src/lightning/fabric/utilities/distributed.py:385

        super().__init__(_DatasetSamplerWrapper(sampler), *args, **kwargs)

    @override
    def __iter__(self) -> Iterator:
        self.dataset.reset()
        return (self.dataset[index] for index in super().__iter__())

    @override
    def set_epoch(self, epoch: int) -> None:
        super().set_epoch(epoch)
        # Forward set_epoch to the original sampler if it supports it
        original_sampler = self.dataset._sampler
        if hasattr(original_sampler, "set_epoch") and callable(original_sampler.set_epoch):
            original_sampler.set_epoch(epoch)


def _suggested_max_num_threads(num_processes: int = 1) -> int:
    if num_processes < 1:
        raise ValueError(f"`num_processes` should be >= 1, got {num_processes}.")
    return max(1, _num_cpus_available() // num_processes)


def _set_num_threads_if_needed(num_processes: int = 1) -> None:
    if "OMP_NUM_THREADS" not in os.environ:
        num_threads = _suggested_max_num_threads(num_processes)
        torch.set_num_threads(num_threads)
        os.environ["OMP_NUM_THREADS"] = str(num_threads)


def _distributed_is_initialized() -> bool:
    # `is_initialized` is only defined conditionally
    # https://github.com/pytorch/pytorch/blob/v2.1.0/torch/distributed/__init__.py#L25
    # this might happen to MacOS builds from source (default) or any build from source that sets `USE_DISTRIBUTED=0`
    return torch.distributed.is_available() and torch.distributed.is_initialized()


class _InfiniteBarrier:

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Ensure num_processes is at least 1 (e.g. `max(1, num_processes)`) before launching Fabric/Trainer or torchrun
2. Check how num_processes is derived: world_size, device count, or env vars must not yield 0
3. If calling the private helper directly, validate the argument first

Example fix

// before
fabric = Fabric(num_processes=0)

// after
from lightning_fabric.utilities import _suggested_max_num_threads
num_processes = max(1, int(os.environ.get("WORLD_SIZE", 1)))
fabric = Fabric(num_processes=num_processes)

num_processes = max(1, int(num_processes))
assert num_processes >= 1

When it happens

Trigger: Calling `seed_everything`/Fabric or Trainer internals that invoke `_set_num_threads_if_needed(num_processes)` with 0 or a negative number; passing `num_processes=0` to a Fabric/launcher utility; computing num_processes as `world_size - 1` or from an env var that resolves to 0.

Common situations: Off-by-one when deriving device/process counts, passing a computed `num_processes` that is 0 on single-process runs, misconfigured launcher env vars (e.g. empty WORLD_SIZE coerced to 0).

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Invalid mode. Has to be min or max, found {self.mode}
• Received multiple values for {', '.join(duplicated_plugin_ke
• Received both `precision={precision_input}` and `plugins={se
• accelerator set through both strategy class and accelerator
• precision set through both strategy class and plugins, choos

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/00c18f227c9f7d2c. Report an issue: GitHub.