{"record":{"id":"c44490f88df164b4","repo":"rohitg00/ai-engineering-from-scratch","slug":"inverse-sqrt-warmup-requires-warmup-steps-0","errorCode":null,"errorMessage":"inverse-sqrt warmup requires warmup_steps > 0","messagePattern":"inverse-sqrt warmup requires warmup_steps > 0","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phases/19-capstone-projects/44-cosine-lr-warmup/code/main.py","lineNumber":282,"sourceCode":"        if step >= self.warmup_steps:\n            return self.lr_max\n        return self.lr_max * (step / self.warmup_steps)\n\n\n@dataclass\nclass InverseSqrtWarmup:\n    \"\"\"Linear warmup followed by an inverse-square-root decay.\n\n    Decays as lr_max * sqrt(warmup_steps / step) for step > warmup_steps. Used\n    historically in transformer training and useful as a comparison baseline.\n    \"\"\"\n\n    warmup_steps: int\n    lr_max: float\n\n    def __post_init__(self) -> None:\n        if self.warmup_steps <= 0:\n            raise ValueError(\"inverse-sqrt warmup requires warmup_steps > 0\")\n        if self.lr_max <= 0:\n            raise ValueError(\"lr_max must be positive\")\n\n    def lr(self, step: int) -> float:\n        if step < 0:\n            raise ValueError(f\"step must be non-negative, got {step}\")\n        if step <= self.warmup_steps:\n            return self.lr_max * (step / self.warmup_steps)\n        return self.lr_max * math.sqrt(self.warmup_steps / step)\n\n\n@dataclass\nclass EWMA:\n    \"\"\"Exponentially weighted moving average of a scalar, useful for grad-norm smoothing.\"\"\"\n\n    beta: float\n    value: float = 0.0\n    initialized: bool = False","sourceCodeStart":264,"sourceCodeEnd":300,"githubUrl":"https://github.com/rohitg00/ai-engineering-from-scratch/blob/39ea8a1c6d0b61f071226eff7ede4d4105fed820/phases/19-capstone-projects/44-cosine-lr-warmup/code/main.py#L264-L300","documentation":"Error \"inverse-sqrt warmup requires warmup_steps > 0\" thrown in rohitg00/ai-engineering-from-scratch.","triggerScenarios":"Thrown at phases/19-capstone-projects/44-cosine-lr-warmup/code/main.py:282 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":[],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"39ea8a1c6d0b61f071226eff7ede4d4105fed820","analyzedAt":"2026-08-26T03:13:46.626Z","schemaVersion":2},"datasetVersion":"2026-08-26T07:17:17.940Z"}