jax-ml/jax · error · ValueError
Reverse-mode differentiation does not work for…
Error message
Reverse-mode differentiation does not work for lax.while_loop or lax.fori_loop with dynamic start/stop values. Try using lax.scan, or using fori_loop with static start/stop.
What it means
Reverse-mode differentiation (grad/vjp) of a while_loop/fori_loop requires knowing the number of iterations ahead of time so JAX can build the linearized transpose. When the loop bounds (or the while condition's trip count) are dynamic (traced) values, JAX cannot reverse the loop and raises this error, suggesting alternatives that have known trip counts.
Solutions
- Use lax.fori_loop with static (Python int or concrete) start/stop so JAX specializes the trip count
- Replace the dynamic loop with lax.scan if you can express the fixed maximum iterations with masking (e.g. run to max_len and mask inactive steps)
- Recompute the bound outside jit and pass it as a static argument (functools.partial or static_argnums)
- Wrap the whole loop in a custom_vjp defining explicit forward/backward rules if differentiation through a truly dynamic loop is required
Example fix
// before def f(x, n): return lax.fori_loop(0, n, lambda i, a: a + x[i], 0.0) jax.grad(lambda x: f(x, n_tapped))(x) // after def f(x, n): n = int(n) # pass as static_argnums instead if inside jit return lax.fori_loop(0, n, lambda i, a: a + x[i], 0.0)
Defensive patterns
Strategy: validation
Validate before calling
import jax
def bounds_are_static(lower, upper):
return jax.core.is_concrete(lower) and jax.core.is_concrete(upper)
# or: type(lower) is int and type(upper) is int outside jit Type guard
def differentiable_loop_safe(lower, upper):
return isinstance(lower, (int,)) and isinstance(upper, (int,)) or \
(jax.core.is_concrete(lower) and jax.core.is_concrete(upper)) Prevention
- Pass loop bounds as static_argnums when they determine trip counts
- Default to lax.scan with a static length + masking for dynamic-length data
- Wrap dynamic loops needing gradients in jax.custom_vjp with manual rules
When it happens
Trigger: Calling jax.grad (or vjp/linearize) on a function using lax.while_loop with a dynamic condition, or lax.fori_loop whose lower/upper are traced arrays (e.g. computed from inputs or inside jit with dynamic values), rather than Python ints / static values.
Common situations: Migrating a Python for loop to fori_loop inside jit where bounds come from data (e.g. sequence lengths, number of steps from a tensor); using while(cond_fn) loops in optimization/solver code and then trying to differentiate through them.
Related errors
- at , got fwd output type which doesn't match primal output…
- Body jaxpr has consts. If you see this error, please open…
- Buffer callbacks do not support transpose. Please use…
- Called with a float0 array. float0s do not support any…
- Called with a float0 at position . float0s do not support…
AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27).
Data as JSON: /api/errors/356c1f5d1d00a22f.
Report an issue: GitHub.
Appendix: source
Thrown at jax/_src/lax/control_flow/loops.py:2076
params_known['cond_nconsts'])
eqn_known = pe.new_jaxpr_eqn(ins_known, out_binders_known, while_p,
params_known, effects_known, eqn.source_info,
eqn.ctx)
# Typecheck known eqn.
_while_loop_abstract_eval(
*[v.aval for v in eqn_known.invars], cond_jaxpr=cond_jaxpr_known,
body_jaxpr=body_jaxpr_known, body_nconsts=params_known['body_nconsts'],
cond_nconsts=params_known['cond_nconsts'])
# Staged eqn is same as input eqn.
eqn_staged = eqn
unks_out = carry_uk
inst_out = [True] * len(unks_out)
return eqn_known, eqn_staged, unks_out, inst_out, new_inst
def _while_transpose_error(*_, **kwargs):
raise ValueError("Reverse-mode differentiation does not work for "
"lax.while_loop or lax.fori_loop with dynamic start/stop values. "
"Try using lax.scan, or using fori_loop with static start/stop.")
# For a while loop with ordered effects in the cond, we need a special
# lowering. Fundamentally, we'd like to rewrite a while loop that looks like
# this:
# ```
# while cond(x):
# x = body(x)
# ```
# into something that looks like this:
# ```
# while True:
# token, pred = cond(token, x)
# if not pred:
# break
# token, x = body(token, x)
# ```View on GitHub (pinned to 1e1c6a8fc0)