unslothai/unsloth · error · ValueError

gradient_accumulation_steps must be >= 1

Error message

gradient_accumulation_steps must be >= 1

What it means

Raised by the training-config validator before any GPU work starts: gradient_accumulation_steps was set below 1. Gradient accumulation splits a large effective batch into micro-batches, so zero or negative accumulation is meaningless and would crash or mis-scale the optimizer later in the trainer. The check exists in validation so a bad request fails cheaply, before resident GPU models are evicted.

Source

Thrown at studio/backend/core/training/diffusion_train_common.py:1014

    # Optional explicit family override; None = detect from base_model. ``resolved_family`` is filled by normalized() with the trainer family that will run.
    model_family: Optional[str] = None
    resolved_family: str = "sdxl"

    def normalized(self) -> "DiffusionLoraConfig":
        """Return a copy with derived/validated fields filled in. Raises ValueError on a
        request that cannot train (bad numbers, or an untrainable base model).

        Also coerces values that arrive as strings/blanks through the Studio config path
        (``learning_rate`` is preserved as a string there; ``hf_token`` defaults to "")."""
        resolved_family = resolve_trainable_family(self.base_model, self.model_family)
        if self.train_steps < 1:
            raise ValueError("train_steps must be >= 1")
        if not 0 <= int(self.num_epochs) <= 1000:
            raise ValueError("num_epochs must be between 0 and 1000 (0 uses train_steps)")
        if self.train_batch_size < 1:
            raise ValueError("train_batch_size must be >= 1")
        if self.gradient_accumulation_steps < 1:
            raise ValueError("gradient_accumulation_steps must be >= 1")
        if self.lora_rank < 1:
            raise ValueError("lora_rank must be >= 1")
        if self.lora_alpha is not None and self.lora_alpha < 1:
            raise ValueError(
                "lora_alpha must be >= 1 (a zero/negative alpha scales the adapter to nothing)"
            )
        if self.resolution < 64 or self.resolution % 8 != 0:
            raise ValueError("resolution must be a multiple of 8 and >= 64")
        # A video family's VAE compresses space by 32, so an off-grid resolution changes the
        # latent geometry silently. Refuse it here, before the GPU models are evicted.
        if (
            resolved_family in TRAINABLE_VIDEO_FAMILIES
            and self.resolution % _VIDEO_RESOLUTION_MULTIPLE != 0
        ):
            raise ValueError(
                f"'{resolved_family}' trains at a resolution that is a multiple of "
                f"{_VIDEO_RESOLUTION_MULTIPLE} (its VAE compresses space by that factor); "
                f"got {self.resolution}."

View on GitHub (pinned to 203007d190)

Solutions

  1. Set gradient_accumulation_steps to 1 or higher (1 = no accumulation).
  2. If the value arrives as a string through the Studio config path, make sure it is a non-empty numeric string that parses to >= 1.
  3. If computing it dynamically (e.g. effective_batch / micro_batch), clamp to at least 1: max(1, computed).
  4. Audit the request payload right before submission and fail fast in your own code with a clearer message.

Example fix

# before
config = TrainConfig(gradient_accumulation_steps=0)

# after
config = TrainConfig(gradient_accumulation_steps=1)
Defensive patterns

Strategy: validation

Validate before calling

def check_gradient_accumulation_steps(v) -> int:
    n = int(v) if v not in (None, "") else 1
    if n < 1:
        raise ValueError(f"gradient_accumulation_steps must be >= 1, got {v!r}")
    return n

config.gradient_accumulation_steps = check_gradient_accumulation_steps(config.gradient_accumulation_steps)

Type guard

def is_valid_gradient_accumulation_steps(v) -> bool:
    try:
        return int(v) >= 1
    except (TypeError, ValueError):
        return False

Try / catch

try:
    session.submit_training(config)
except ValueError as e:
    if "gradient_accumulation_steps" in str(e):
        config.gradient_accumulation_steps = 1
        session.submit_training(config)
    else:
        raise

Prevention

When it happens

Trigger: Submitting a training request (Studio config path or direct API call) with gradient_accumulation_steps = 0, a negative number, or a string like "0"/"" that the Studio config coercion turns into 0. Also happens when a UI form leaves the field defaulted to 0 or a computation like batch_size // micro_batch yields 0.

Common situations: Config files or YAML hand-edited with a 0 to 'disable' accumulation; frontend forms whose numeric input defaults to 0; arithmetic that derives accumulation steps from other values and floors to 0 for small batch sizes.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/966abb2388c180d0. Report an issue: GitHub.