jax-ml/jax · error · NotImplementedError

group_offset is not currently supported in the pallas-triton

Error message

group_offset is not currently supported in the pallas-triton lowering.

What it means

The pallas-triton lowering of ragged_dot_general does not support the group_offset argument, which allows applying only a sub-range of groups. Only full grouped dot semantics are implemented on this backend.

Source

Thrown at jax/_src/lax/pallas_lowerings/gpu/ragged_dot.py:467

def _pallas_ragged_dot_general_impl(
    lhs: Array,
    rhs: Array,
    group_sizes: Array,
    ragged_dot_dimension_numbers: lax.RaggedDotDimensionNumbers,
    precision: lax.PrecisionLike = None,  # TODO(rdyro): Add support for the
                                          # precision parameter in the kernels.
    preferred_element_type: DTypeLike | None = None,
    group_offset: Array | None = None,
    out_sharding=None,
) -> Array:
  if out_sharding is not None:
    raise NotImplementedError(
      "Explicit sharding is not currently supported in the pallas-triton"
      " lowering of ragged_dot_general. You can call this op under shard_map.")

  if group_offset is not None:
    raise NotImplementedError("group_offset is not currently supported in the "
                              "pallas-triton lowering.")

  mode, lhs_ragged_dim = lax._ragged_dot_mode_and_dim(
      lhs.ndim, ragged_dot_dimension_numbers)
  (l_contract, r_contract), (l_batch, r_batch) = (
      ragged_dot_dimension_numbers.dot_dimension_numbers
  )
  l_contract, r_contract = tuple(l_contract), tuple(r_contract)
  l_batch, r_batch = tuple(l_batch), tuple(r_batch)
  l_noncontract = tuple(lax.remaining(range(lhs.ndim), l_contract, l_batch))
  assert len(l_batch) == len(r_batch)

  if group_sizes.ndim == 1:
    group_sizes = lax.broadcast(group_sizes, [lhs.shape[i] for i in l_batch])

  compute_dtype = np.promote_types(lhs.dtype, rhs.dtype)
  out_dtype = (np.dtype(preferred_element_type)
               if preferred_element_type is not None else None)

View on GitHub (pinned to 1e1c6a8fc0)

Solutions

1. Remove group_offset and process all groups at once (slice the weights/activations manually instead)
2. Pre-slice A and group_sizes so the desired group subrange becomes the full call
3. Use the XLA ragged_dot_general path rather than the pallas-triton lowering if group_offset is required

Example fix

// before
out = ragged_dot_general(lhs, rhs, dims, group_sizes, group_offset=off)

// after
A_sub, gs_sub = A[off:off+n], group_sizes[off:off+n]
out = ragged_dot_general(lhs, A_sub, dims, gs_sub)

if group_offset is not None and backend == 'triton':
    raise SystemExit('slice A/group_sizes manually instead of group_offset')

When it happens

Trigger: Passing a non-None group_offset to ragged_dot_general when lowering through pallas-triton on GPU (e.g. chunked/pipelined MoE group processing).

Common situations: Reusing TPU MoE code that slices expert groups via group_offset; attempting incremental grouped matmul to cap memory on GPU.

Related errors

• Unimplemented group_offset support.
• ragged_dot vmap over any dim but 0 - NYI
• Left eigenvectors are not supported by cusolver
• subset_by_index not supported on CPU and GPU
• This gmm kernel only supports either (m, k) x (g, k, n) -> (

AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27). Data as JSON: /api/errors/e489aaad9bc7798d. Report an issue: GitHub.