huggingface/transformers · warning · ValueError

DebugUnderflowOverflow: aborting after {self.batch_number} b

Error message

DebugUnderflowOverflow: aborting after {self.batch_number} batches due to `abort_after_batch_num={self.abort_after_batch_num}` arg

What it means

DebugUnderflowOverflow supports an abort_after_batch_num argument to stop execution after a chosen batch for interactive debugging. When self.batch_number exceeds that limit, it raises this ValueError deliberately. This is a controlled, requested abort - the exception text mirrors the argument you passed - not a fault in the model or data.

Source

Thrown at src/transformers/debug_utils.py:289

        if trace_mode:
            self.trace_frames()

        if last_frame_of_batch:
            self.batch_start_frame()

        if self.detected_overflow and not trace_mode:
            self.dump_saved_frames()

            # now we can abort, as it's pointless to continue running
            raise ValueError(
                "DebugUnderflowOverflow: inf/nan detected, aborting as there is no point running further. "
                "Please scroll up above this traceback to see the activation values prior to this event."
            )

        # abort after certain batch if requested to do so
        if self.abort_after_batch_num is not None and self.batch_number > self.abort_after_batch_num:
            raise ValueError(
                f"DebugUnderflowOverflow: aborting after {self.batch_number} batches due to"
                f" `abort_after_batch_num={self.abort_after_batch_num}` arg"
            )


def get_abs_min_max(var, ctx):
    abs_var = var.abs()
    return f"{abs_var.min():8.2e} {abs_var.max():8.2e} {ctx}"


def detect_overflow(var, ctx):
    """
    Report whether the tensor contains any `nan` or `inf` entries.

    This is useful for detecting overflows/underflows and best to call right after the function that did some math that
    modified the tensor in question.

    This function contains a few other helper features that you can enable and tweak directly if you want to track

View on GitHub (pinned to a597f97485)

Solutions

  1. If the abort was intentional, inspect state in the debugger/REPL as planned; then raise or remove abort_after_batch_num.
  2. For full runs, detach the debugger: do not instantiate DebugUnderflowOverflow (or set abort_after_batch_num=None).
  3. If you still need overflow detection for the whole run, keep the instance but without abort_after_batch_num.

Example fix

# before
debug_overflow = DebugUnderflowOverflow(model, abort_after_batch_num=5)
trainer.train()  # raises at batch 6 by design

# after: full training, no artificial abort
debug_overflow = DebugUnderflowOverflow(model)  # or remove entirely
trainer.train()
Defensive patterns

Strategy: try-catch

Try / catch

try:
    trainer.train()
except ValueError as e:
    if "abort_after_batch_num" in str(e):
        logger.info("debugger abort reached; continuing without debug hook")
    else:
        raise

Prevention

When it happens

Trigger: Constructing DebugUnderflowOverflow(model, abort_after_batch_num=N) and letting training run past batch N; leaving a debug instance attached from an earlier debugging session.

Common situations: Interactive debugging where you drop into a debugger at a specific batch to inspect weights; forgetting to remove the debug hook before launching a real training run.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/e22e5b6d8338dcf3. Report an issue: GitHub.