Lightning-AI/pytorch-lightning · error · ValueError

`num_nodes` must be a positive integer, but got {num_nodes}.

Error message

`num_nodes` must be a positive integer, but got {num_nodes}.

What it means

Fabric validates `num_nodes` immediately in the connector: it must be a Python int and >= 1. Passing 0, a negative number, a float like 2.0, or a string raises this error before any setup runs.

Source

Thrown at src/lightning/fabric/connector.py:300

                if self._strategy_flag.parallel_devices[0].type == "cpu":
                    if self._accelerator_flag and self._accelerator_flag not in ("auto", "cpu"):
                        raise ValueError(
                            f"CPU parallel_devices set through {self._strategy_flag.__class__.__name__} class,"
                            f" but accelerator set to {self._accelerator_flag}, please choose one device type"
                        )
                    self._accelerator_flag = "cpu"
                if self._strategy_flag.parallel_devices[0].type == "cuda":
                    if self._accelerator_flag and self._accelerator_flag not in ("auto", "cuda", "gpu"):
                        raise ValueError(
                            f"GPU parallel_devices set through {self._strategy_flag.__class__.__name__} class,"
                            f" but accelerator set to {self._accelerator_flag}, please choose one device type"
                        )
                    self._accelerator_flag = "cuda"
                self._parallel_devices = self._strategy_flag.parallel_devices

    def _check_device_config_and_set_final_flags(self, devices: Union[list[int], str, int], num_nodes: int) -> None:
        if not isinstance(num_nodes, int) or num_nodes < 1:
            raise ValueError(f"`num_nodes` must be a positive integer, but got {num_nodes}.")

        self._num_nodes_flag = num_nodes
        self._devices_flag = devices

        if self._devices_flag in ([], 0, "0"):
            accelerator_name = (
                self._accelerator_flag.__class__.__qualname__
                if isinstance(self._accelerator_flag, Accelerator)
                else self._accelerator_flag
            )
            raise ValueError(
                f"`Fabric(devices={self._devices_flag!r})` value is not a valid input"
                f" using {accelerator_name} accelerator."
            )

    @staticmethod
    def _choose_auto_accelerator() -> str:
        """Choose the accelerator type (str) based on availability when ``accelerator='auto'``."""

View on GitHub (pinned to 9fed5c27d2)

Solutions

  1. Pass an explicit positive int, e.g. num_nodes=1
  2. Cast config-sourced values: num_nodes=int(raw_value) and validate >= 1 before constructing Fabric
  3. Fix upstream computations that can produce 0 or negative node counts

Example fix

# before
fabric = Fabric(num_nodes=os.environ.get("NUM_NODES", "1"))

# after
num_nodes = int(os.environ.get("NUM_NODES", "1"))
fabric = Fabric(num_nodes=max(1, num_nodes))
Defensive patterns

Strategy: type-guard

Validate before calling

def coerce_num_nodes(v):
    n = int(v)
    if n < 1:
        raise ValueError(f"num_nodes must be >= 1, got {n}")
    return n

num_nodes = coerce_num_nodes(os.environ.get("NUM_NODES", 1))
fabric = Fabric(num_nodes=num_nodes)

Type guard

def is_valid_num_nodes(v) -> bool:
    return isinstance(v, int) and not isinstance(v, bool) and v >= 1

Prevention

When it happens

Trigger: Fabric(num_nodes=0), Fabric(num_nodes=-1), Fabric(num_nodes="2"), or Fabric(num_nodes=2.0) (a bool True passes isinstance but 2 nodes values from config parsing often don't).

Common situations: Reading num_nodes from env vars or YAML/JSON config as a string; arithmetic that yields 0 (e.g. world_size // devices_per_node with tiny values); typos in argparse defaults like num_nodes="1".

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors


AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/dc3564412f52670c. Report an issue: GitHub.