jax-ml/jax · error · ValueError

process_id and num_processes must be nonnegative, with proce

Error message

process_id and num_processes must be nonnegative, with process_id < num_processes. Got process_id={process_id}, num_processes={num_processes}.

What it means

Beyond type checks, initialize validates the range 0 <= process_id < num_processes. A rank outside the world size (negative, or >= num_processes) raises this ValueError reporting both values.

Source

Thrown at jax/_src/distributed.py:149

              cluster_detection_method,
              initialization_timeout,
          )
      )

    if coordinator_address is None:
      raise ValueError('coordinator_address should be defined.')
    if num_processes is None:
      raise ValueError('Number of processes must be defined.')
    if process_id is None:
      raise ValueError('The process id of the current process must be defined.')
    if not isinstance(process_id, int):
      raise TypeError("process_id must be a nonnegative int. "
                      f"Got process_id={process_id} of type {type(process_id)}.")
    if not isinstance(num_processes, int):
      raise TypeError("num_processes must be a positive int. "
                      f"Got num_processes={num_processes} of type {type(num_processes)}.")
    if not (0 <= process_id < num_processes):
      raise ValueError("process_id and num_processes must be nonnegative, with process_id < num_processes. "
                       f"Got process_id={process_id}, num_processes={num_processes}.")

    self.coordinator_address = coordinator_address

    # The default value of [::]:port tells the coordinator to bind to all
    # available addresses on the same port as coordinator_address.
    default_coordinator_bind_address = '[::]:' + coordinator_address.rsplit(':', 1)[1]
    coordinator_bind_address = (coordinator_bind_address or
                                os.environ.get('JAX_COORDINATOR_BIND_ADDRESS',
                                               default_coordinator_bind_address))
    if coordinator_bind_address is None:
      raise ValueError('coordinator_bind_address should be defined.')

    if local_device_ids:
      visible_devices = ','.join(str(x) for x in local_device_ids)
      logger.info('JAX distributed initialized with visible devices: %s', visible_devices)
      config.update("jax_cuda_visible_devices", visible_devices)
      config.update("jax_rocm_visible_devices", visible_devices)

View on GitHub (pinned to 1e1c6a8fc0)

Solutions

1. Ensure rank and world size come from the same launcher source: int(os.environ['RANK']) and int(os.environ['WORLD_SIZE'])
2. Add a launch-time assertion: assert 0 <= rank < world_size
3. Unset stale job env vars between runs (env -i or fresh shells)

Example fix

# before
rank, world = int(os.environ['RANK']), 8  # hardcoded, RANK may be 8
# after
rank = int(os.environ['RANK']); world = int(os.environ['WORLD_SIZE'])
assert 0 <= rank < world
jax.distributed.initialize(coordinator_address=addr,
    num_processes=world, process_id=rank)

rank, world = int(os.environ['RANK']), int(os.environ['WORLD_SIZE'])
assert 0 <= rank < world

When it happens

Trigger: process_id=8 with num_processes=8; negative ranks; inconsistent rank/world-size env vars, e.g. RANK from a different launcher than WORLD_SIZE.

Common situations: Mixing launcher env vars (torchrun RANK with a hardcoded world size); off-by-one in manual rank assignment; env leakage between sequential jobs in CI.

Related errors

• Mapped away dimension of inputs passed to vmap should be sha
• The 'sharding' attribute is not available on {self._error_re
• The is_fully_addressable property was called on {self._error
• mTLS for the JAX distributed service requires jaxlib 0.11.2
• coordinator_address should be defined.

AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27). Data as JSON: /api/errors/6d72197133444ff1. Report an issue: GitHub.