jax-ml/jax · error · ValueError

Can't make a peer reference into a multicast reference.

Error message

Can't make a peer reference into a multicast reference.

What it means

multicast_ref refuses references that already carry a PeerMemRef transform: a reference cannot target one specific peer device and simultaneously multicast across a collective axis.

Source

Thrown at jax/_src/pallas/mosaic_gpu/core.py:1117

def multicast_ref(
    ref: _Ref,
    collective_axes: Hashable | tuple[Hashable, ...],
) -> pallas_core.TransformedRef:
  """Return a multicast reference for cross-device operations.

  Args:
    ref: The reference to transform.
    collective_axes: The JAX mesh axes indicating the devices to operate on.
  """
  if not isinstance(collective_axes, tuple):
    collective_axes = (collective_axes,)
  if not isinstance(ref, pallas_core.TransformedRef):
    if not isinstance(jax_core.typeof(ref), state_types.AbstractRef):
      raise TypeError("ref must be a reference")
    ref = pallas_core.TransformedRef(ref, transforms=())
  if any(isinstance(t, PeerMemRef) for t in ref.transforms):
    raise ValueError("Can't make a peer reference into a multicast reference.")
  return pallas_core.TransformedRef(
      ref.ref, (*ref.transforms, MulticastRef(collective_axes)),
  )


def transpose_ref(
    ref: pallas_core.TransformedRef | Any,
    permutation: tuple[int, ...],
) -> pallas_core.TransformedRef:
  assert hasattr(ref, "memory_space")
  if ref.memory_space == MemorySpace.TMEM:
    raise ValueError("Can't transpose a TMEM reference.")
  return ref.transpose(permutation)


@tree_util.register_dataclass
@dataclasses.dataclass(frozen=True)
class ExtractAliasedRef(state_types.Transform):

View on GitHub (pinned to 1e1c6a8fc0)

Solutions

1. Use separate references for peer stores and multicast stores
2. Create the multicast ref from the underlying ref before wrapping with remote_ref on a different ref
3. Restructure the store sequence so transforms are never composed

Example fix

// before
p = remote_ref(out_ref, 1)
m = multicast_ref(p, 'dev')

// after
m = multicast_ref(out_ref, 'dev')

assert not any(isinstance(t, PeerMemRef) for t in ref.transforms), 'already a peer ref'

When it happens

Trigger: Calling multicast_ref(remote_ref(...)) — composing peer and multicast transforms on the same reference during a collective store.

Common situations: TPU collective kernels mixing remote (single-device) and multicast (multi-device) stores on the same output reference.

Related errors

• Can't make a multicast reference into a peer reference.
• ref must be a reference
• Only collective_axes that include all JAX device mesh axes
• packed cannot be specified if layout is specified.
• packed, collective and layout arguments are only supported f

AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27). Data as JSON: /api/errors/4fc4344143fe6abc. Report an issue: GitHub.