{"record":{"id":"aa3ba71f1e80ebee","repo":"jax-ml/jax","slug":"all-gather-reduced-s-input-cannot-be-reduced-acros","errorCode":null,"errorMessage":"all_gather_reduced's input cannot be reduced across the axis_name provided. Got x={x_aval.str_short(True)} and {axis_name=}","messagePattern":"all_gather_reduced's input cannot be reduced across the axis_name provided\\. Got x=(.+?) and (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"jax/_src/lax/parallel.py","lineNumber":2596,"sourceCode":"  return tree_util.tree_map(bind, x)\n\nall_gather_reduced_p = core.Primitive('all_gather_reduced')\n\ndef _all_gather_reduced_effectful_abstract_eval(\n    x_aval, *, all_gather_dimension, axis_name, axis_size, tiled\n):\n  _check_axis_names(axis_name, 'all_gather_reduced')\n  if not x_aval.mat.varying:\n    raise ValueError('all_gather_reduced only accepts inputs that are'\n                     f' varying. Got {x_aval.str_short(True)}')\n  # If the intersection between x.mat.varying and axis_name is empty, error\n  if not (x_aval.mat.varying & set(axis_name)):\n    raise ValueError(\n        'all_gather_reduced is a Varying -> Reduced collective. This means '\n        f'that the {axis_name=} passed to `all_gather_reduced` must be present '\n        f'in jax.typeof(x).mat.varying={x_aval.mat.varying}')\n  if x_aval.mat.reduced & set(axis_name):\n    raise ValueError(\n        \"all_gather_reduced's input cannot be reduced across the axis_name\"\n        f\" provided. Got x={x_aval.str_short(True)} and {axis_name=}\")\n\n  new_shape = list(x_aval.shape)\n  if tiled:\n    new_shape[all_gather_dimension] *= axis_size\n  else:\n    new_shape.insert(all_gather_dimension, axis_size)\n\n  if x_aval.mat.unreduced:\n    check_unreduced_kind('all_gather_reduced', x_aval.mat, UnreducedKind.sum)\n  new_reduced = x_aval.mat.reduced | frozenset(axis_name)\n  out_vma = frozenset(v for v in x_aval.mat.varying if v not in axis_name)\n  out_mat = x_aval.mat.update(varying=out_vma, reduced=new_reduced)\n  return (x_aval.update(shape=new_shape, manual_axis_type=out_mat),\n          {*map(core.NamedAxisEffect, axis_name)})\nall_gather_reduced_p.def_effectful_abstract_eval(\n    _all_gather_reduced_effectful_abstract_eval)","sourceCodeStart":2578,"sourceCodeEnd":2614,"githubUrl":"https://github.com/jax-ml/jax/blob/1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb/jax/_src/lax/parallel.py#L2578-L2614","documentation":"`all_gather_reduced` requires its input to not already be reduced along the requested axis (`x.mat.reduced` must not intersect `axis_name`). An already-reduced value has already participated in a sum across that axis, so gathering it again would double-count semantics. The check in the abstract eval catches inconsistent collective pipelines early.","triggerScenarios":"Feeding the output of a psum-like reduced collective (which marks the axis as reduced) directly into `all_gather_reduced` on the same axis name.","commonSituations":"Chaining collectives (psum then all_gather) while porting manual SPMD code; misunderstanding that reduced-ness is tracked in the type and persists through operations.","solutions":["Restructure so the input is varying (not reduced) along axis_name, e.g. apply pvary to a fresh invariant value","Insert a barrier/reset by re-annotating with pvary on a non-reduced value","Use a different collective that accepts reduced inputs if that matches your intent"],"exampleFix":"// before\nr = lax.psum(x, 'i')           # r.mat.reduced contains 'i'\ny = lax.all_gather_reduced(r, axis_name='i', ...)\n// after\nr = lax.psum(x, 'i')\ny = lax.all_gather_reduced(lax.pvary(r, 'i'), axis_name='i', ...)  # only if re-varying is intended","handlingStrategy":"validation","validationCode":"if jax.typeof(x).mat.reduced & set(axis_name):\n    x = lax.pvary(x, axis_name)  # re-annotate if re-gathering intended\ny = lax.all_gather_reduced(x, axis_name=axis_name, ...)","typeGuard":"def not_reduced_on(x, axis_name) -> bool:\n    return not (jax.typeof(x).mat.reduced & set(axis_name))","tryCatchPattern":"try:\n    y = lax.all_gather_reduced(x, axis_name=axis_name, ...)\nexcept ValueError as e:\n    if 'cannot be reduced across' in str(e):\n        raise RuntimeError(f'pipeline bug: {axis_name} already reduced') from e\n    raise","preventionTips":["Track collective state per axis in code review","Unit-test collective pipelines with a tiny mesh to catch state conflicts"],"tags":["jax","collectives","mesh","reduced","all-gather"],"backgroundTag":"jax-mat-axis-state-mismatch","analyzedSha":"1e1c6a8fc06dfcd1247076ec5cae4640cea5d7bb","analyzedAt":"2026-08-27T09:53:25.647Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}