unslothai/unsloth · error · ValueError

save_total_limit must be >= 0 (0 keeps every checkpoint)

Error message

save_total_limit must be >= 0 (0 keeps every checkpoint)

What it means

The validator rejected a negative save_total_limit. save_total_limit caps how many periodic checkpoints are kept on disk; 0 is the documented 'keep every checkpoint' value (no pruning), so negatives have no defined meaning and are refused in validation before GPU resources are touched.

Source

Thrown at studio/backend/core/training/diffusion_train_common.py:1064

            raise ValueError(
                f"lr_scheduler must be one of {', '.join(sorted(_LR_SCHEDULERS))}; "
                f"got {self.lr_scheduler!r}"
            )
        if not 1 <= int(self.cache_variants) <= 16:
            raise ValueError("cache_variants must be between 1 and 16")
        # Checkpointing knobs. Rejected here, before the route evicts resident GPU models, rather than deep in the loop.
        try:
            save_steps = int(self.save_steps or 0)
            save_total_limit = int(self.save_total_limit or 0)
        except (TypeError, ValueError) as exc:
            raise ValueError(
                f"save_steps / save_total_limit must be whole numbers, got "
                f"{self.save_steps!r} / {self.save_total_limit!r}"
            ) from exc
        if save_steps < 0:
            raise ValueError("save_steps must be >= 0 (0 disables periodic checkpoints)")
        if save_total_limit < 0:
            raise ValueError("save_total_limit must be >= 0 (0 keeps every checkpoint)")
        # A blank resume path (the Studio default when the field is present but unset) means "fresh run", not the outputs root.
        resume_from_checkpoint = (
            str(self.resume_from_checkpoint).strip()
            if self.resume_from_checkpoint is not None
            else ""
        ) or None
        # The H3 loop does not checkpoint: it neither writes a resume bundle nor restores one.
        # Accepting these two silently was the dangerous part -- a caller handing over a resume
        # bundle got a FRESH optimization that then overwrote the outputs it was meant to
        # continue, and one asking for periodic saves got none, both discovered only after an
        # expensive run. Refuse in validation, where it costs nothing, until the loop supports it.
        if resolved_family in CHECKPOINTLESS_FAMILIES:
            if resume_from_checkpoint:
                raise ValueError(
                    f"resume_from_checkpoint is not supported for {resolved_family}: its trainer "
                    f"writes no checkpoint bundle, so there is nothing to continue from and the "
                    f"run would silently start over and overwrite its output. Start a fresh run."
                )

View on GitHub (pinned to 203007d190)

Solutions

1. Use 0 to keep every checkpoint — this trainer's 'unlimited' value.
2. Use a positive count (e.g. 3) to keep only the N newest checkpoints.
3. Grep your config templates for -1 sentinels when migrating from other trainers.

Example fix

# before (HuggingFace-style 'keep all')
config = TrainConfig(save_total_limit=-1)

# after
config = TrainConfig(save_total_limit=0)

def check_save_total_limit(v) -> int:
    n = int(v or 0)
    if n < 0:
        raise ValueError(f"save_total_limit must be >= 0 (0 keeps every checkpoint), got {v!r}")
    return n

def is_valid_save_total_limit(v) -> bool:
    try:
        return int(v or 0) >= 0
    except (TypeError, ValueError):
        return False

try:
    session.submit_training(config)
except ValueError as e:
    if "save_total_limit must be >= 0" in str(e):
        config.save_total_limit = 0  # keep every checkpoint
        session.submit_training(config)
    else:
        raise

When it happens

Trigger: save_total_limit=-1 passed directly or as the string '-1'. Very commonly caused by porting configs from HuggingFace Accelerate/Trainer, where save_total_limit=-1 historically means 'keep everything'.

Common situations: Copy-pasting a HuggingFace training script's arguments block into this Studio's config; assuming -1 is a universal 'unlimited' sentinel.

Related errors

• save_steps / save_total_limit must be whole numbers, got {se
• save_steps must be >= 0 (0 disables periodic checkpoints)
• save_steps is not supported for {resolved_family}: its train
• gradient_accumulation_steps must be >= 1
• lora_rank must be >= 1

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b93b845968561c75. Report an issue: GitHub.