rohitg00/ai-engineering-from-scratch · error · ValueError

device_type must be 'cpu' or 'cuda', got {device_type}

Error message

device_type must be 'cpu' or 'cuda', got {device_type}

What it means

Error "device_type must be 'cpu' or 'cuda', got {device_type}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/45-gradient-clipping-amp/code/main.py:152

      4. scaler.unscale_(optimizer).
      5. Gradient finiteness check; non-finite grad skips optimizer step.
      6. Clip to max_norm.
      7. scaler.step(optimizer); scaler.update().
    """

    def __init__(
        self,
        model: nn.Module,
        lr: float = 1e-2,
        max_norm: float = DEFAULT_MAX_NORM,
        device_type: str = DEFAULT_DEVICE,
        weight_decay: float = 0.01,
        amp_dtype: torch.dtype | None = None,
    ) -> None:
        if max_norm <= 0:
            raise ValueError("max_norm must be positive")
        if device_type not in ("cpu", "cuda"):
            raise ValueError(f"device_type must be 'cpu' or 'cuda', got {device_type}")
        self.model = model
        self.max_norm = max_norm
        self.device_type = device_type
        self.optimizer = torch.optim.AdamW(
            model.parameters(),
            lr=lr,
            weight_decay=weight_decay,
        )
        scaler_enabled = device_type == "cuda"
        self.scaler = torch.amp.GradScaler(device_type, enabled=scaler_enabled)
        if amp_dtype is None:
            amp_dtype = torch.bfloat16 if device_type == "cpu" else torch.float16
        self.amp_dtype = amp_dtype
        self.global_step = 0
        self._log: list[StepLog] = []
        self._skip_log: list[SkipLog] = []
        self._loss_fn: Callable[[torch.Tensor, torch.Tensor], torch.Tensor] = nn.functional.mse_loss

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/45-gradient-clipping-amp/code/main.py:152 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/3432a8911114fa87. Report an issue: GitHub.