{"record":{"id":"8b807695f64120c4","repo":"karpathy/nanochat","slug":"formula-timed-out-after-duration-seconds","errorCode":null,"errorMessage":"'{formula}': timed out after {duration} seconds","messagePattern":"'(.+?)': timed out after (.+?) seconds","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"warning","filePath":"nanochat/engine.py","lineNumber":28,"sourceCode":"\nThe whole thing is made as efficient as possible.\n\"\"\"\n\nimport torch\nimport torch.nn.functional as F\nimport signal\nimport warnings\nfrom contextlib import contextmanager\nfrom collections import deque\nfrom nanochat.common import compute_init, autodetect_device_type, COMPUTE_DTYPE\nfrom nanochat.checkpoint_manager import load_model\n\n# -----------------------------------------------------------------------------\n# Calculator tool helpers\n@contextmanager\ndef timeout(duration, formula):\n    def timeout_handler(signum, frame):\n        raise Exception(f\"'{formula}': timed out after {duration} seconds\")\n\n    signal.signal(signal.SIGALRM, timeout_handler)\n    signal.alarm(duration)\n    yield\n    signal.alarm(0)\n\ndef eval_with_timeout(formula, max_time=3):\n    try:\n        with timeout(max_time, formula):\n            with warnings.catch_warnings():\n                warnings.simplefilter(\"ignore\", SyntaxWarning)\n                return eval(formula, {\"__builtins__\": {}}, {})\n    except Exception as e:\n        signal.alarm(0)\n        # print(f\"Warning: Failed to eval {formula}, exception: {e}\") # it's ok ignore wrong calculator usage\n        return None\n\ndef use_calculator(expr):","sourceCodeStart":10,"sourceCodeEnd":46,"githubUrl":"https://github.com/karpathy/nanochat/blob/92d63d4e8bb4df75c3b71618f31ddde2378b2bcd/nanochat/engine.py#L10-L46","documentation":"The calculator tool in nanochat/engine.py evaluates arithmetic expressions from model output via Python `eval` (sandboxed to no builtins) guarded by a SIGALRM-based `timeout` context manager. If the expression takes longer than `max_time` seconds (default 3), the signal handler fires mid-eval and raises a generic Exception with the offending formula. This is a deliberate guard against pathological expressions (e.g. huge exponentiation like `9**9**9`) hanging the chat engine.","triggerScenarios":"The language model emits a calculator tool call whose expression is computationally explosive — `10**100**2`, very large factorials via repeated multiplication, or expressions producing gigantic integers — so eval exceeds 3 seconds of CPU time.","commonSituations":"Small chat models that malformed expressions (nested exponent towers); adversarial or prompt-injected user input asking for huge powers; models trained rarely producing degenerate arithmetic.","solutions":["No code fix needed for callers: the timeout is intended behavior; the engine catches it and reports tool failure to the model. Verify the calling code wraps calculator evaluation in try/except and feeds the error back as tool output.","If legitimate expressions time out, raise max_time in `eval_with_timeout(expr, max_time=...)`.","For SFT data generation, filter/skip such formulas so the model rarely produces them.","Note the limitation: SIGALRM only works in the main thread of a Unix process — running eval_with_timeout from a non-main thread will not fire the alarm."],"exampleFix":"// not applicable (internal timeout guard; behavior is by design)","handlingStrategy":"try-catch","validationCode":"import re\n# cheap pre-filter: reject exponent towers / huge literals before eval\nif re.search(r\"\\*\\*.*\\*\\*\", formula) or re.search(r\"\\d{10,}\", formula):\n    return \"error: expression rejected (too expensive)\"","typeGuard":null,"tryCatchPattern":"try:\n    result = eval_with_timeout(formula)\nexcept Exception as e:  # timeout raises generic Exception with 'timed out' in message\n    result = f\"error: {e}\"  # feed back to the model as tool output","preventionTips":["Always catch the timeout exception and return the error string as tool output so the model can recover.","Only run eval_with_timeout from the main thread (SIGALRM does not fire in worker threads).","Pre-filter obviously explosive expressions (nested **) before eval.","Keep max_time small (default 3s) to bound worst-case latency."],"tags":["nanochat","calculator","timeout","tool-use","signal"],"backgroundTag":null,"analyzedSha":"92d63d4e8bb4df75c3b71618f31ddde2378b2bcd","analyzedAt":"2026-08-15T03:11:54.371Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}