{"record":{"id":"f12dec87a46d0a68","repo":"huggingface/pytorch-image-models","slug":"lr-as-a-tensor-is-not-supported-for-capturable-fal","errorCode":null,"errorMessage":"lr as a Tensor is not supported for capturable=False and foreach=True","messagePattern":"lr as a Tensor is not supported for capturable=False and foreach=True","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/optim/adopt.py","lineNumber":83,"sourceCode":"            self,\n            params: ParamsT,\n            lr: Union[float, Tensor] = 1e-3,\n            betas: Tuple[float, float] = (0.9, 0.9999),\n            eps: float = 1e-6,\n            clip_exp: Optional[float] = 0.333,\n            weight_decay: float = 0.0,\n            decoupled: bool = False,\n            corrected_weight_decay: bool = False,\n            *,\n            caution: bool = False,\n            foreach: Optional[bool] = False,\n            maximize: bool = False,\n            capturable: bool = False,\n            differentiable: bool = False,\n    ):\n        if isinstance(lr, Tensor):\n            if foreach and not capturable:\n                raise ValueError(\n                    \"lr as a Tensor is not supported for capturable=False and foreach=True\"\n                )\n            if lr.numel() != 1:\n                raise ValueError(\"Tensor lr must be 1-element\")\n        if not 0.0 <= lr:\n            raise ValueError(f\"Invalid learning rate: {lr}\")\n        if not 0.0 <= eps:\n            raise ValueError(f\"Invalid epsilon value: {eps}\")\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 0: {betas[0]}\")\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 1: {betas[1]}\")\n        if not 0.0 <= weight_decay:\n            raise ValueError(f\"Invalid weight_decay value: {weight_decay}\")\n\n        defaults = dict(\n            lr=lr,\n            betas=betas,","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/optim/adopt.py#L65-L101","documentation":"ADOPT's constructor rejects a tensor learning rate when foreach=True and capturable=False, because the multi-tensor foreach path without CUDA-graph capturability cannot read lr from a tensor without a device synchronization that breaks the fused implementation.","triggerScenarios":"Constructing timm.optim.Adopt(params, lr=torch.tensor(1e-3), foreach=True, capturable=False).","commonSituations":"Using a tensor lr for lr-scheduling tricks or per-device lr handling while leaving foreach at its default (which resolves to True when supported by the device).","solutions":["Pass capturable=True if you genuinely need a tensor lr with foreach","Use a plain float lr instead of a tensor","Set foreach=False to keep the tensor lr in the single-tensor path"],"exampleFix":"# before\nopt = timm.optim.Adopt(model.parameters(), lr=torch.tensor(1e-3), foreach=True)\n# after\nopt = timm.optim.Adopt(model.parameters(), lr=1e-3)","handlingStrategy":"validation","validationCode":"if isinstance(lr, torch.Tensor):\n    assert not (foreach and not capturable), 'tensor lr requires capturable=True or foreach=False'\n    assert lr.numel() == 1","typeGuard":"def valid_adopt_lr(lr, foreach: bool, capturable: bool) -> bool:\n    if not isinstance(lr, torch.Tensor):\n        return lr >= 0.0\n    return lr.numel() == 1 and (capturable or not foreach)","tryCatchPattern":null,"preventionTips":["Default to float lrs; only use tensor lr with capturable=True","Set foreach explicitly when using non-default lr types"],"tags":["optimizer","adopt","tensor-lr","foreach","capturable"],"backgroundTag":"optimizer-tensor-lr-unsupported","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}