unslothai/unsloth · error · ValueError

lora_alpha must be >= 1 (a zero/negative alpha scales the ad

Error message

lora_alpha must be >= 1 (a zero/negative alpha scales the adapter to nothing)

What it means

The validator rejected lora_alpha < 1 (when alpha is not None). In the standard LoRA scaling formula the adapter output is multiplied by alpha/rank, so a zero or negative alpha scales the adapter's contribution to nothing (or flips its sign) — training would run at full cost while learning an effectively disabled adapter. The guard catches this before the expensive run starts.

Source

Thrown at studio/backend/core/training/diffusion_train_common.py:1018

    def normalized(self) -> "DiffusionLoraConfig":
        """Return a copy with derived/validated fields filled in. Raises ValueError on a
        request that cannot train (bad numbers, or an untrainable base model).

        Also coerces values that arrive as strings/blanks through the Studio config path
        (``learning_rate`` is preserved as a string there; ``hf_token`` defaults to "")."""
        resolved_family = resolve_trainable_family(self.base_model, self.model_family)
        if self.train_steps < 1:
            raise ValueError("train_steps must be >= 1")
        if not 0 <= int(self.num_epochs) <= 1000:
            raise ValueError("num_epochs must be between 0 and 1000 (0 uses train_steps)")
        if self.train_batch_size < 1:
            raise ValueError("train_batch_size must be >= 1")
        if self.gradient_accumulation_steps < 1:
            raise ValueError("gradient_accumulation_steps must be >= 1")
        if self.lora_rank < 1:
            raise ValueError("lora_rank must be >= 1")
        if self.lora_alpha is not None and self.lora_alpha < 1:
            raise ValueError(
                "lora_alpha must be >= 1 (a zero/negative alpha scales the adapter to nothing)"
            )
        if self.resolution < 64 or self.resolution % 8 != 0:
            raise ValueError("resolution must be a multiple of 8 and >= 64")
        # A video family's VAE compresses space by 32, so an off-grid resolution changes the
        # latent geometry silently. Refuse it here, before the GPU models are evicted.
        if (
            resolved_family in TRAINABLE_VIDEO_FAMILIES
            and self.resolution % _VIDEO_RESOLUTION_MULTIPLE != 0
        ):
            raise ValueError(
                f"'{resolved_family}' trains at a resolution that is a multiple of "
                f"{_VIDEO_RESOLUTION_MULTIPLE} (its VAE compresses space by that factor); "
                f"got {self.resolution}."
            )
        if self.mixed_precision not in ("bf16", "fp16", "no"):
            raise ValueError("mixed_precision must be one of bf16 / fp16 / no")
        # torch.manual_seed unpacks int64/uint64, so anything wider raises inside the trainer, after eviction. Catch it here.

View on GitHub (pinned to 203007d190)

Solutions

1. Set lora_alpha to >= 1 (a common convention is alpha = 2 * rank or alpha = rank).
2. If you want no alpha scaling, pass lora_alpha=None rather than 0.
3. Check any formula that computes alpha (e.g. rank * ratio) and clamp it to at least 1.

Example fix

# before
config = TrainConfig(lora_rank=16, lora_alpha=0)

# after
config = TrainConfig(lora_rank=16, lora_alpha=32)

def check_lora_alpha(v, rank: int) -> int | None:
    if v is None:
        return None
    a = int(v)
    if a < 1:
        raise ValueError(f"lora_alpha must be >= 1, got {v!r}; pass None to omit")
    return a

def is_valid_lora_alpha(v) -> bool:
    if v is None:
        return True
    try:
        return int(v) >= 1
    except (TypeError, ValueError):
        return False

try:
    session.submit_training(config)
except ValueError as e:
    if "lora_alpha" in str(e):
        config.lora_alpha = None  # omit and let the trainer pick its default
        session.submit_training(config)
    else:
        raise

When it happens

Trigger: A training request with lora_alpha = 0 or a negative number (e.g. -4), or a coerced string that parses to such a value. Passing lora_alpha=None is fine — only explicit bad numbers are rejected.

Common situations: Configs copied from experiments where alpha was deliberately set to 0 to ablate LoRA; scaling formulas like alpha = rank * scale that round to 0 for tiny scale values; sweep grids including 0.

Related errors

• gradient_accumulation_steps must be >= 1
• lora_rank must be >= 1
• resolution must be a multiple of 8 and >= 64
• mixed_precision must be one of bf16 / fp16 / no
• seed must fit in torch's 64-bit range

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b1f9d44071a9cb62. Report an issue: GitHub.