{"record":{"id":"e94f46780a8feb0d","repo":"jax-ml/jax","slug":"expected-exactly-one-collective-axis-got-collect","errorCode":null,"errorMessage":"Expected exactly one collective axis, got {collective_axes=}","messagePattern":"Expected exactly one collective axis, got (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/_src/pallas/mosaic_gpu/primitives.py","lineNumber":985,"sourceCode":"        lowering._resolve_cluster_axis(ctx.module_ctx.axis_names, axis)\n        for axis in collective_axes\n    )\n\n  is_leader_tracked_copy = collective and leader_tracked is not None\n  dst_ty = ir.MemRefType(dst.type)\n  bits = math.prod(dst_ty.shape) * mgpu.bitwidth(dst_ty.element_type)\n  if bits % 8:\n    raise ValueError(\n        f\"Can only transfer integer bytes (shape={dst_ty.shape},\"\n        f\" dtype={dst_ty.element_type})\"\n    )\n  bytes = bits // 8\n\n  if is_leader_tracked_copy:\n    # Leader receives the completion messages from both CTAs.\n    bytes *= 2\n    if len(collective) != 1:\n      raise ValueError(\n          f\"Expected exactly one collective axis, got {collective_axes=}\"\n      )\n    if math.prod(ctx.launch_ctx.cluster_size) != 2:\n      raise NotImplementedError(\n          \"Partitioned loads only supported for clusters of size 2. Got\"\n          f\" cluster size {ctx.launch_ctx.cluster_size}.\"\n      )\n\n  # TMA is only available on Hopper and newer. On older architectures we fall\n  # back to the cp.async implementation.\n  if is_cp_async := mgpu.utils.get_arch().major < 9:\n    if barrier is not None:\n      raise ValueError(\n          \"copy_gmem_to_smem with a barrier is only supported Hopper and newer\"\n          \" GPUs, which use the TMA implementation\"\n      )\n    if collective_axes is not None:\n      raise ValueError(\"Only the TMA implementation supports collective copies\")","sourceCodeStart":967,"sourceCodeEnd":1003,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/_src/pallas/mosaic_gpu/primitives.py#L967-L1003","documentation":"For leader-tracked collective copies, both CTAs in the cluster send completion messages to the leader, so the code requires exactly one collective axis (len(collective) == 1); otherwise the accounting of doubled bytes is ambiguous and ValueError is raised.","triggerScenarios":"copy_gmem_to_smem with leader_tracked set and collective_axes containing zero or more than one axis names.","commonSituations":"Reusing a multi-axis mesh tuple for collective_axes when the design only partitions over one axis; passing collective_axes=() with leader_tracked still enabled.","solutions":["Pass exactly one collective axis name, e.g. collective_axes=('data',)","Drop leader_tracked if you don't need leader completion tracking","Restructure the copy into separate per-axis collective copies"],"exampleFix":"# before\ncopy_gmem_to_smem(src, smem, collective_axes=('data','model'), leader_tracked=True)\n# after\ncopy_gmem_to_smem(src, smem, collective_axes=('data',), leader_tracked=True)","handlingStrategy":"validation","validationCode":"if leader_tracked is not None:\n    assert len(collective_axes) == 1, 'leader_tracked requires exactly one collective axis'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Use exactly one collective axis name with leader_tracked copies"],"tags":["jax","pallas","mosaic-gpu","collective-axes","leader-tracked"],"backgroundTag":"invalid-argument-combination","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}