{"record":{"id":"b93b845968561c75","repo":"unslothai/unsloth","slug":"save-total-limit-must-be-0-0-keeps-every-check","errorCode":null,"errorMessage":"save_total_limit must be >= 0 (0 keeps every checkpoint)","messagePattern":"save_total_limit must be >= 0 \\(0 keeps every checkpoint\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_train_common.py","lineNumber":1064,"sourceCode":"            raise ValueError(\n                f\"lr_scheduler must be one of {', '.join(sorted(_LR_SCHEDULERS))}; \"\n                f\"got {self.lr_scheduler!r}\"\n            )\n        if not 1 <= int(self.cache_variants) <= 16:\n            raise ValueError(\"cache_variants must be between 1 and 16\")\n        # Checkpointing knobs. Rejected here, before the route evicts resident GPU models, rather than deep in the loop.\n        try:\n            save_steps = int(self.save_steps or 0)\n            save_total_limit = int(self.save_total_limit or 0)\n        except (TypeError, ValueError) as exc:\n            raise ValueError(\n                f\"save_steps / save_total_limit must be whole numbers, got \"\n                f\"{self.save_steps!r} / {self.save_total_limit!r}\"\n            ) from exc\n        if save_steps < 0:\n            raise ValueError(\"save_steps must be >= 0 (0 disables periodic checkpoints)\")\n        if save_total_limit < 0:\n            raise ValueError(\"save_total_limit must be >= 0 (0 keeps every checkpoint)\")\n        # A blank resume path (the Studio default when the field is present but unset) means \"fresh run\", not the outputs root.\n        resume_from_checkpoint = (\n            str(self.resume_from_checkpoint).strip()\n            if self.resume_from_checkpoint is not None\n            else \"\"\n        ) or None\n        # The H3 loop does not checkpoint: it neither writes a resume bundle nor restores one.\n        # Accepting these two silently was the dangerous part -- a caller handing over a resume\n        # bundle got a FRESH optimization that then overwrote the outputs it was meant to\n        # continue, and one asking for periodic saves got none, both discovered only after an\n        # expensive run. Refuse in validation, where it costs nothing, until the loop supports it.\n        if resolved_family in CHECKPOINTLESS_FAMILIES:\n            if resume_from_checkpoint:\n                raise ValueError(\n                    f\"resume_from_checkpoint is not supported for {resolved_family}: its trainer \"\n                    f\"writes no checkpoint bundle, so there is nothing to continue from and the \"\n                    f\"run would silently start over and overwrite its output. Start a fresh run.\"\n                )","sourceCodeStart":1046,"sourceCodeEnd":1082,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_train_common.py#L1046-L1082","documentation":"The validator rejected a negative save_total_limit. save_total_limit caps how many periodic checkpoints are kept on disk; 0 is the documented 'keep every checkpoint' value (no pruning), so negatives have no defined meaning and are refused in validation before GPU resources are touched.","triggerScenarios":"save_total_limit=-1 passed directly or as the string '-1'. Very commonly caused by porting configs from HuggingFace Accelerate/Trainer, where save_total_limit=-1 historically means 'keep everything'.","commonSituations":"Copy-pasting a HuggingFace training script's arguments block into this Studio's config; assuming -1 is a universal 'unlimited' sentinel.","solutions":["Use 0 to keep every checkpoint — this trainer's 'unlimited' value.","Use a positive count (e.g. 3) to keep only the N newest checkpoints.","Grep your config templates for -1 sentinels when migrating from other trainers."],"exampleFix":"# before (HuggingFace-style 'keep all')\nconfig = TrainConfig(save_total_limit=-1)\n\n# after\nconfig = TrainConfig(save_total_limit=0)","handlingStrategy":"validation","validationCode":"def check_save_total_limit(v) -> int:\n    n = int(v or 0)\n    if n < 0:\n        raise ValueError(f\"save_total_limit must be >= 0 (0 keeps every checkpoint), got {v!r}\")\n    return n","typeGuard":"def is_valid_save_total_limit(v) -> bool:\n    try:\n        return int(v or 0) >= 0\n    except (TypeError, ValueError):\n        return False","tryCatchPattern":"try:\n    session.submit_training(config)\nexcept ValueError as e:\n    if \"save_total_limit must be >= 0\" in str(e):\n        config.save_total_limit = 0  # keep every checkpoint\n        session.submit_training(config)\n    else:\n        raise","preventionTips":["Memorize this trainer's convention: 0 = keep all, N = keep newest N; negatives are never valid.","Translate HuggingFace's -1 to 0 in any config migration script.","Document the sentinel mapping next to your model of the trainer's config schema."],"tags":["training","checkpointing","configuration","validation"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}