{"record":{"id":"6598cefba52da822","repo":"karpathy/nanochat","slug":"no-checkpoints-found-in-checkpoint-dir","errorCode":null,"errorMessage":"No checkpoints found in {checkpoint_dir}","messagePattern":"No checkpoints found in (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"nanochat/checkpoint_manager.py","lineNumber":141,"sourceCode":"    candidates = []\n    for model_tag in model_tags:\n        match = re.match(r\"d(\\d+)\", model_tag)\n        if match:\n            model_depth = int(match.group(1))\n            candidates.append((model_depth, model_tag))\n    if candidates:\n        candidates.sort(key=lambda x: x[0], reverse=True)\n        return candidates[0][1]\n    # 2) if that failed, take the most recently updated model:\n    model_tags.sort(key=lambda x: os.path.getmtime(os.path.join(checkpoints_dir, x)), reverse=True)\n    return model_tags[0]\n\n\ndef find_last_step(checkpoint_dir):\n    # Look into checkpoint_dir and find model_<step>.pt with the highest step\n    checkpoint_files = [f for f in os.listdir(checkpoint_dir) if re.search(r'model_(\\d+)\\.pt$', f)]\n    if not checkpoint_files:\n        raise FileNotFoundError(f\"No checkpoints found in {checkpoint_dir}\")\n    last_step = max(int(f.split(\"_\")[-1].split(\".\")[0]) for f in checkpoint_files)\n    return last_step\n\n# -----------------------------------------------------------------------------\n# convenience functions that take into account nanochat's directory structure\n\ndef load_model_from_dir(checkpoints_dir, device, phase, model_tag=None, step=None):\n    if model_tag is None:\n        # guess the model tag by defaulting to the largest model\n        model_tag = find_largest_model(checkpoints_dir)\n        log0(f\"No model tag provided, guessing model tag: {model_tag}\")\n    checkpoint_dir = os.path.join(checkpoints_dir, model_tag)\n    if step is None:\n        # guess the step by defaulting to the last step\n        step = find_last_step(checkpoint_dir)\n    assert step is not None, f\"No checkpoints found in {checkpoint_dir}\"\n    # build the model\n    log0(f\"Loading model from {checkpoint_dir} with step {step}\")","sourceCodeStart":123,"sourceCodeEnd":159,"githubUrl":"https://github.com/karpathy/nanochat/blob/92d63d4e8bb4df75c3b71618f31ddde2378b2bcd/nanochat/checkpoint_manager.py#L123-L159","documentation":"`find_last_step(checkpoint_dir)` scans a single model-tag directory for files matching `model_<step>.pt` and returns the highest step. If none match — the directory is empty or contains only optimizer/meta files — FileNotFoundError is raised. This value is used to resume/load the latest checkpoint of a chosen model.","triggerScenarios":"Calling `find_last_step` on a model-tag directory with no `model_*.pt` files: training crashed before the first checkpoint save, checkpoints were partially cleaned, or the path points to a directory holding only `optimizer_<step>.pt`/meta files.","commonSituations":"A training run interrupted before its first periodic save; someone deleted the .pt model files but left other artifacts; passing the parent checkpoints dir instead of the model-tag subdir.","solutions":["List the model-tag directory and confirm at least one file named like `model_0001234.pt` exists.","If only optimizer/meta files remain, restore the matching model_*.pt from backup or restart training.","Make sure you pass the model-tag subdirectory (e.g. assets/checkpoints/d24), not the parent checkpoints dir (which contains no model files and would also trip this error)."],"exampleFix":"# before\nstep = find_last_step(\"assets/checkpoints\")  # wrong level: no model_*.pt here\n\n# after\nstep = find_last_step(\"assets/checkpoints/d24\")  # dir containing model_0001234.pt","handlingStrategy":"validation","validationCode":"import os, re\nckpts = [f for f in os.listdir(checkpoint_dir) if re.search(r'model_(\\d+)\\.pt$', f)]\nif not ckpts:\n    raise SystemExit(f\"No model_<step>.pt files in {checkpoint_dir}; cannot resume.\")\nlast_step = max(int(f.split('_')[-1].split('.')[0]) for f in ckpts)","typeGuard":null,"tryCatchPattern":"try:\n    step = find_last_step(checkpoint_dir)\nexcept FileNotFoundError:\n    print(\"No model checkpoints; starting from scratch or aborting\")\n    raise","preventionTips":["Ensure training saves the first checkpoint early (small initial save interval) so resume always has a model_<step>.pt.","Keep model_*.pt and optimizer_*.pt files in sync when cleaning up checkpoint dirs.","Pass the model-tag subdirectory, not the parent checkpoints dir."],"tags":["nanochat","checkpoints","resume","filesystem"],"backgroundTag":null,"analyzedSha":"92d63d4e8bb4df75c3b71618f31ddde2378b2bcd","analyzedAt":"2026-08-15T03:11:54.371Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}