jax-ml/jax · warning

TRACER_LEAK_DEBUGGER_WARNING

Error message

TRACER_LEAK_DEBUGGER_WARNING

What it means

JAX detected leaked tracers (JAX's intermediate values during tracing) and, because the current thread is being traced by a Python debugger (pydev), it warns that the debugger itself may be responsible. Debuggers keep frame references alive, which keeps Tracer objects from being garbage collected, producing false-positive leak reports.

Source

Thrown at jax/_src/core.py:1618

@contextmanager
def ensure_no_leaks(trace:Trace):
  yield
  trace.invalidate()
  if config.check_tracer_leaks.value:
    trace_ref = trace._weakref
    del trace
    live_trace = trace_ref()
    if live_trace is not None:
      leaked_tracers = maybe_find_leaked_tracers(live_trace)
      if leaked_tracers:
        raise leaked_tracer_error("trace", live_trace, leaked_tracers)


def maybe_find_leaked_tracers(trace: Trace) -> list[Tracer]:
  """Find the leaked tracers holding a reference to the Trace
  """
  if not getattr(threading.current_thread(), 'pydev_do_not_trace', True):
    warnings.warn(TRACER_LEAK_DEBUGGER_WARNING)
  # Trigger garbage collection to filter out unreachable objects that are alive
  # only due to cyclical dependencies. (We don't care about unreachable leaked
  # tracers since they can't interact with user code and cause a problem.)
  gc.collect()
  tracers = list(filter(lambda x: isinstance(x, Tracer), gc.get_referrers(trace)))
  return tracers

def leaked_tracer_error(name: str, t, tracers: list[Tracer]) -> Exception:
  assert tracers
  why = partial(_why_alive, {id(tracers)})
  msgs = []
  for tracer in tracers:  # not a genexpr: it'd be gc-visible and self-report
    chain = why(tracer)
    label = f'<{type(tracer).__name__} {id(tracer)}>'
    chain += ''.join(f'\n{label} is referred to by {h}' for h in
                     _held_in_frame_locals(tracer, {id(tracers)}))
    if not chain:
      chain = (f'\n{label} has no referrers visible to the gc module; it may '

View on GitHub (pinned to 1e1c6a8fc0)

Solutions

1. Ignore the warning if it appears only while a debugger is attached — rerun without the debugger to confirm.
2. Remove breakpoints that capture intermediate Tracer values, or del tracer references before leaving the function.
3. If leaks persist without a debugger, find where the Tracer escapes (e.g. stored in a global/attribute) and remove the retention.

Example fix

# before
global _cache
_cache = x  # x is a Tracer leaked out of a jitted trace
# after
# don't retain tracers; materialize first
_cache = jax.device_put(x) if outside trace else None

import threading
running_under_debugger = not getattr(threading.current_thread(), 'pydev_do_not_trace', True)

with warnings.catch_warnings():
    warnings.filterwarnings('ignore', message='.*TRACER LEAK.*')
    fn()  # during debugged runs

When it happens

Trigger: Running under a debugger (PyCharm/VSCode via pydev/pydevd) with JAX tracing that yields leaked-tracer diagnostics; maybe_find_leaked_tracers fires and sees pydev_do_not_trace is falsy on the thread, meaning the debugger is active.

Common situations: Debugging a jitted function in PyCharm or VSCode; breakpoints holding references to tracer values; check_leaks tests failing only when a debugger is attached.

Related errors

• the VJP function was applied before restoring its not-saveab
• {str(exc)}
• Value of type {type(self)} is not convertible to hex.
• The on_device_size_in_bytes() method was called on {self._er
• The traceback property was called on {self._error_repr()}.{s

AI-assisted analysis of jax-ml/jax@1e1c6a8fc0 (2026-08-27). Data as JSON: /api/errors/20887f5ae2102081. Report an issue: GitHub.