{"record":{"id":"b1f9d44071a9cb62","repo":"unslothai/unsloth","slug":"lora-alpha-must-be-1-a-zero-negative-alpha-sca","errorCode":null,"errorMessage":"lora_alpha must be >= 1 (a zero/negative alpha scales the adapter to nothing)","messagePattern":"lora_alpha must be >= 1 \\(a zero/negative alpha scales the adapter to nothing\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_train_common.py","lineNumber":1018,"sourceCode":"    def normalized(self) -> \"DiffusionLoraConfig\":\n        \"\"\"Return a copy with derived/validated fields filled in. Raises ValueError on a\n        request that cannot train (bad numbers, or an untrainable base model).\n\n        Also coerces values that arrive as strings/blanks through the Studio config path\n        (``learning_rate`` is preserved as a string there; ``hf_token`` defaults to \"\").\"\"\"\n        resolved_family = resolve_trainable_family(self.base_model, self.model_family)\n        if self.train_steps < 1:\n            raise ValueError(\"train_steps must be >= 1\")\n        if not 0 <= int(self.num_epochs) <= 1000:\n            raise ValueError(\"num_epochs must be between 0 and 1000 (0 uses train_steps)\")\n        if self.train_batch_size < 1:\n            raise ValueError(\"train_batch_size must be >= 1\")\n        if self.gradient_accumulation_steps < 1:\n            raise ValueError(\"gradient_accumulation_steps must be >= 1\")\n        if self.lora_rank < 1:\n            raise ValueError(\"lora_rank must be >= 1\")\n        if self.lora_alpha is not None and self.lora_alpha < 1:\n            raise ValueError(\n                \"lora_alpha must be >= 1 (a zero/negative alpha scales the adapter to nothing)\"\n            )\n        if self.resolution < 64 or self.resolution % 8 != 0:\n            raise ValueError(\"resolution must be a multiple of 8 and >= 64\")\n        # A video family's VAE compresses space by 32, so an off-grid resolution changes the\n        # latent geometry silently. Refuse it here, before the GPU models are evicted.\n        if (\n            resolved_family in TRAINABLE_VIDEO_FAMILIES\n            and self.resolution % _VIDEO_RESOLUTION_MULTIPLE != 0\n        ):\n            raise ValueError(\n                f\"'{resolved_family}' trains at a resolution that is a multiple of \"\n                f\"{_VIDEO_RESOLUTION_MULTIPLE} (its VAE compresses space by that factor); \"\n                f\"got {self.resolution}.\"\n            )\n        if self.mixed_precision not in (\"bf16\", \"fp16\", \"no\"):\n            raise ValueError(\"mixed_precision must be one of bf16 / fp16 / no\")\n        # torch.manual_seed unpacks int64/uint64, so anything wider raises inside the trainer, after eviction. Catch it here.","sourceCodeStart":1000,"sourceCodeEnd":1036,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_train_common.py#L1000-L1036","documentation":"The validator rejected lora_alpha < 1 (when alpha is not None). In the standard LoRA scaling formula the adapter output is multiplied by alpha/rank, so a zero or negative alpha scales the adapter's contribution to nothing (or flips its sign) — training would run at full cost while learning an effectively disabled adapter. The guard catches this before the expensive run starts.","triggerScenarios":"A training request with lora_alpha = 0 or a negative number (e.g. -4), or a coerced string that parses to such a value. Passing lora_alpha=None is fine — only explicit bad numbers are rejected.","commonSituations":"Configs copied from experiments where alpha was deliberately set to 0 to ablate LoRA; scaling formulas like alpha = rank * scale that round to 0 for tiny scale values; sweep grids including 0.","solutions":["Set lora_alpha to >= 1 (a common convention is alpha = 2 * rank or alpha = rank).","If you want no alpha scaling, pass lora_alpha=None rather than 0.","Check any formula that computes alpha (e.g. rank * ratio) and clamp it to at least 1."],"exampleFix":"# before\nconfig = TrainConfig(lora_rank=16, lora_alpha=0)\n\n# after\nconfig = TrainConfig(lora_rank=16, lora_alpha=32)","handlingStrategy":"validation","validationCode":"def check_lora_alpha(v, rank: int) -> int | None:\n    if v is None:\n        return None\n    a = int(v)\n    if a < 1:\n        raise ValueError(f\"lora_alpha must be >= 1, got {v!r}; pass None to omit\")\n    return a","typeGuard":"def is_valid_lora_alpha(v) -> bool:\n    if v is None:\n        return True\n    try:\n        return int(v) >= 1\n    except (TypeError, ValueError):\n        return False","tryCatchPattern":"try:\n    session.submit_training(config)\nexcept ValueError as e:\n    if \"lora_alpha\" in str(e):\n        config.lora_alpha = None  # omit and let the trainer pick its default\n        session.submit_training(config)\n    else:\n        raise","preventionTips":["Pass lora_alpha=None rather than 0 to disable explicit alpha scaling.","Prefer the alpha = 2 * rank convention; it can never hit the bound.","Exclude 0 from alpha sweeps — it ablates to a no-op adapter at full training cost."],"tags":["training","lora","configuration","validation"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}