karpathy/nanochat · warning · Exception

'{formula}': timed out after {duration} seconds

Error message

'{formula}': timed out after {duration} seconds

What it means

The calculator tool in nanochat/engine.py evaluates arithmetic expressions from model output via Python `eval` (sandboxed to no builtins) guarded by a SIGALRM-based `timeout` context manager. If the expression takes longer than `max_time` seconds (default 3), the signal handler fires mid-eval and raises a generic Exception with the offending formula. This is a deliberate guard against pathological expressions (e.g. huge exponentiation like `9**9**9`) hanging the chat engine.

Source

Thrown at nanochat/engine.py:28

The whole thing is made as efficient as possible.
"""

import torch
import torch.nn.functional as F
import signal
import warnings
from contextlib import contextmanager
from collections import deque
from nanochat.common import compute_init, autodetect_device_type, COMPUTE_DTYPE
from nanochat.checkpoint_manager import load_model

# -----------------------------------------------------------------------------
# Calculator tool helpers
@contextmanager
def timeout(duration, formula):
    def timeout_handler(signum, frame):
        raise Exception(f"'{formula}': timed out after {duration} seconds")

    signal.signal(signal.SIGALRM, timeout_handler)
    signal.alarm(duration)
    yield
    signal.alarm(0)

def eval_with_timeout(formula, max_time=3):
    try:
        with timeout(max_time, formula):
            with warnings.catch_warnings():
                warnings.simplefilter("ignore", SyntaxWarning)
                return eval(formula, {"__builtins__": {}}, {})
    except Exception as e:
        signal.alarm(0)
        # print(f"Warning: Failed to eval {formula}, exception: {e}") # it's ok ignore wrong calculator usage
        return None

def use_calculator(expr):

View on GitHub (pinned to 92d63d4e8b)

Solutions

1. No code fix needed for callers: the timeout is intended behavior; the engine catches it and reports tool failure to the model. Verify the calling code wraps calculator evaluation in try/except and feeds the error back as tool output.
2. If legitimate expressions time out, raise max_time in `eval_with_timeout(expr, max_time=...)`.
3. For SFT data generation, filter/skip such formulas so the model rarely produces them.
4. Note the limitation: SIGALRM only works in the main thread of a Unix process — running eval_with_timeout from a non-main thread will not fire the alarm.

Example fix

// not applicable (internal timeout guard; behavior is by design)

import re
# cheap pre-filter: reject exponent towers / huge literals before eval
if re.search(r"\*\*.*\*\*", formula) or re.search(r"\d{10,}", formula):
    return "error: expression rejected (too expensive)"

try:
    result = eval_with_timeout(formula)
except Exception as e:  # timeout raises generic Exception with 'timed out' in message
    result = f"error: {e}"  # feed back to the model as tool output

When it happens

Trigger: The language model emits a calculator tool call whose expression is computationally explosive — `10**100**2`, very large factorials via repeated multiplication, or expressions producing gigantic integers — so eval exceeds 3 seconds of CPU time.

Common situations: Small chat models that malformed expressions (nested exponent towers); adversarial or prompt-injected user input asking for huge powers; models trained rarely producing degenerate arithmetic.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Unknown part type: {part['type']}
• Unknown dataset tag: {dataset_tag}
• No checkpoints found in {checkpoints_dir}
• No checkpoints found in {checkpoint_dir}
• Unsupported task type: {task_type}

AI-assisted analysis of karpathy/nanochat@92d63d4e8b (2026-08-15). Data as JSON: /api/errors/8b807695f64120c4. Report an issue: GitHub.