{"record":{"id":"e22e5b6d8338dcf3","repo":"huggingface/transformers","slug":"debugunderflowoverflow-aborting-after-self-batch","errorCode":null,"errorMessage":"DebugUnderflowOverflow: aborting after {self.batch_number} batches due to `abort_after_batch_num={self.abort_after_batch_num}` arg","messagePattern":"DebugUnderflowOverflow: aborting after (.+?) batches due to `abort_after_batch_num=(.+?)` arg","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"warning","filePath":"src/transformers/debug_utils.py","lineNumber":289,"sourceCode":"\n        if trace_mode:\n            self.trace_frames()\n\n        if last_frame_of_batch:\n            self.batch_start_frame()\n\n        if self.detected_overflow and not trace_mode:\n            self.dump_saved_frames()\n\n            # now we can abort, as it's pointless to continue running\n            raise ValueError(\n                \"DebugUnderflowOverflow: inf/nan detected, aborting as there is no point running further. \"\n                \"Please scroll up above this traceback to see the activation values prior to this event.\"\n            )\n\n        # abort after certain batch if requested to do so\n        if self.abort_after_batch_num is not None and self.batch_number > self.abort_after_batch_num:\n            raise ValueError(\n                f\"DebugUnderflowOverflow: aborting after {self.batch_number} batches due to\"\n                f\" `abort_after_batch_num={self.abort_after_batch_num}` arg\"\n            )\n\n\ndef get_abs_min_max(var, ctx):\n    abs_var = var.abs()\n    return f\"{abs_var.min():8.2e} {abs_var.max():8.2e} {ctx}\"\n\n\ndef detect_overflow(var, ctx):\n    \"\"\"\n    Report whether the tensor contains any `nan` or `inf` entries.\n\n    This is useful for detecting overflows/underflows and best to call right after the function that did some math that\n    modified the tensor in question.\n\n    This function contains a few other helper features that you can enable and tweak directly if you want to track","sourceCodeStart":271,"sourceCodeEnd":307,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/debug_utils.py#L271-L307","documentation":"DebugUnderflowOverflow supports an abort_after_batch_num argument to stop execution after a chosen batch for interactive debugging. When self.batch_number exceeds that limit, it raises this ValueError deliberately. This is a controlled, requested abort - the exception text mirrors the argument you passed - not a fault in the model or data.","triggerScenarios":"Constructing DebugUnderflowOverflow(model, abort_after_batch_num=N) and letting training run past batch N; leaving a debug instance attached from an earlier debugging session.","commonSituations":"Interactive debugging where you drop into a debugger at a specific batch to inspect weights; forgetting to remove the debug hook before launching a real training run.","solutions":["If the abort was intentional, inspect state in the debugger/REPL as planned; then raise or remove abort_after_batch_num.","For full runs, detach the debugger: do not instantiate DebugUnderflowOverflow (or set abort_after_batch_num=None).","If you still need overflow detection for the whole run, keep the instance but without abort_after_batch_num."],"exampleFix":"# before\ndebug_overflow = DebugUnderflowOverflow(model, abort_after_batch_num=5)\ntrainer.train()  # raises at batch 6 by design\n\n# after: full training, no artificial abort\ndebug_overflow = DebugUnderflowOverflow(model)  # or remove entirely\ntrainer.train()","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    trainer.train()\nexcept ValueError as e:\n    if \"abort_after_batch_num\" in str(e):\n        logger.info(\"debugger abort reached; continuing without debug hook\")\n    else:\n        raise","preventionTips":["Keep DebugUnderflowOverflow instantiation behind a DEBUG flag.","Set abort_after_batch_num=None (or omit) for full training runs.","Strip debug hooks from launch scripts before long jobs."],"tags":["debugging","intentional-abort","training-loop"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}