tensorflow/models · error · ValueError

When used with `mirrored`, valid values for all_reduce_alg a

Error message

When used with `mirrored`, valid values for all_reduce_alg are [`nccl`, `hierarchical_copy`].  Supplied value: {}

What it means

Error "When used with `mirrored`, valid values for all_reduce_alg are [`nccl`, `hierarchical_copy`]. Supplied value: {}" thrown in tensorflow/models.

Source

Thrown at official/common/distribute_utils.py:68

  Args:
    all_reduce_alg: a string specifying which cross device op to pick, or None.
    num_packs: an integer specifying number of packs for the cross device op.

  Returns:
    tf.distribute.CrossDeviceOps object or None.

  Raises:
    ValueError: if `all_reduce_alg` not in [None, "nccl", "hierarchical_copy"].
  """
  if all_reduce_alg is None:
    return None
  mirrored_all_reduce_options = {
      "nccl": tf.distribute.NcclAllReduce,
      "hierarchical_copy": tf.distribute.HierarchicalCopyAllReduce
  }
  if all_reduce_alg not in mirrored_all_reduce_options:
    raise ValueError(
        "When used with `mirrored`, valid values for all_reduce_alg are "
        "[`nccl`, `hierarchical_copy`].  Supplied value: {}".format(
            all_reduce_alg))
  cross_device_ops_class = mirrored_all_reduce_options[all_reduce_alg]
  return cross_device_ops_class(num_packs=num_packs)


def tpu_initialize(tpu_address):
  """Initializes TPU for TF 2.x training.

  Args:
    tpu_address: string, bns address of master TPU worker.

  Returns:
    A TPUClusterResolver.
  """
  cluster_resolver = tf.distribute.cluster_resolver.TPUClusterResolver(
      tpu=tpu_address)

View on GitHub (pinned to e006f5f0d5)

When it happens

Trigger: Thrown at official/common/distribute_utils.py:68 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of tensorflow/models@e006f5f0d5 (2026-08-24). Data as JSON: /api/errors/d4cbaff7c7dce19f. Report an issue: GitHub.