keras-team/keras · error · ValueError

If `num_processes` is greater than `num_model_replicas`, `nu

Error message

If `num_processes` is greater than `num_model_replicas`, `num_processes` must be divisible by `num_model_replicas`. Got num_processes={self.num_processes}, num_model_replicas={self.num_model_replicas}.

What it means

Error "If `num_processes` is greater than `num_model_replicas`, `num_processes` must be divisible by `num_model_replicas`. Got num_processes={self.num_processes}, num_model_replicas={self.num_model_replicas}." thrown in keras-team/keras.

Source

Thrown at keras/src/distribution/distribution_lib.py:645

        kwargs.pop("device_mesh", None)
        if layout_map is None:
            raise ValueError("You must specify a layout_map argument.")
        if not isinstance(layout_map, LayoutMap):
            raise ValueError(
                "Argument `layout_map` must be a `LayoutMap` instance. "
                f"Received: layout_map={layout_map}"
            )
        device_mesh = layout_map.device_mesh
        batch_dim_name = batch_dim_name or device_mesh.axis_names[0]
        super().__init__(device_mesh, batch_dim_name, auto_shard_dataset)
        self._layout_map = layout_map

        if (
            self._is_multi_process
            and self.num_processes > self.num_model_replicas
            and self.num_processes % self.num_model_replicas != 0
        ):
            raise ValueError(
                "If `num_processes` is greater than `num_model_replicas`, "
                "`num_processes` must be divisible by `num_model_replicas`. "
                f"Got num_processes={self.num_processes}, "
                f"num_model_replicas={self.num_model_replicas}."
            )

    @property
    def num_model_replicas(self):
        mesh_batch_dim_index = self.device_mesh.axis_names.index(
            self.batch_dim_name
        )
        return self.device_mesh.shape[mesh_batch_dim_index]

    def get_data_layout(self, data_shape):
        data_shard_spec = [None] * len(data_shape)
        data_shard_spec[0] = self.batch_dim_name  # Shard on the first dim
        return TensorLayout(data_shard_spec, self.device_mesh)

View on GitHub (pinned to 7a34a03db6)

When it happens

Trigger: Thrown at keras/src/distribution/distribution_lib.py:645 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of keras-team/keras@7a34a03db6 (2026-08-25). Data as JSON: /api/errors/eb0176ebc2fdcf0c. Report an issue: GitHub.