{"record":{"id":"64a628098bd4c97a","repo":"karpathy/nanochat","slug":"unknown-optimizer-kind-group-kind","errorCode":null,"errorMessage":"Unknown optimizer kind: {group['kind']}","messagePattern":"Unknown optimizer kind: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"nanochat/optim.py","lineNumber":446,"sourceCode":"    @torch.no_grad()\n    def step(self):\n        # On a single rank (no multi-rank process group), all communication is skipped\n        if dist.is_available() and dist.is_initialized():\n            rank = dist.get_rank()\n            world_size = dist.get_world_size()\n        else:\n            rank = 0\n            world_size = 1\n\n        # Phase 1: launch all async reduce ops\n        reduce_infos: list[dict] = []\n        for group in self.param_groups:\n            if group['kind'] == 'adamw':\n                reduce_infos.append(self._reduce_adamw(group, world_size))\n            elif group['kind'] == 'muon':\n                reduce_infos.append(self._reduce_muon(group, world_size))\n            else:\n                raise ValueError(f\"Unknown optimizer kind: {group['kind']}\")\n\n        # Phase 2: wait for reduces, compute updates, launch gathers\n        gather_list: list[dict] = []\n        for group, info in zip(self.param_groups, reduce_infos):\n            if group['kind'] == 'adamw':\n                self._compute_adamw(group, info, gather_list, rank, world_size)\n            elif group['kind'] == 'muon':\n                self._compute_muon(group, info, gather_list, rank)\n            else:\n                raise ValueError(f\"Unknown optimizer kind: {group['kind']}\")\n\n        # Phase 3: wait for gathers, copy back\n        self._finish_gathers(gather_list)\n","sourceCodeStart":428,"sourceCodeEnd":460,"githubUrl":"https://github.com/karpathy/nanochat/blob/92d63d4e8bb4df75c3b71618f31ddde2378b2bcd/nanochat/optim.py#L428-L460","documentation":"The fused MuonAdamW optimizer in nanochat/optim.py partitions parameters into groups, each tagged with group['kind'] of 'adamw' or 'muon', and dispatches per-kind reduce operations (all-reduce of grads/optimizer state) in `step()`. A param group whose 'kind' key is missing or misspelled hits the else and raises ValueError during the reduce phase (Phase 1).","triggerScenarios":"Constructing MuonAdamW with a param group dict lacking a 'kind' key (KeyError becomes None via group['kind'] only if key exists — actually a missing key raises KeyError first; the ValueError fires on any wrong value such as 'sgd', 'AdamW' with wrong casing, or 'lion').","commonSituations":"Writing custom training scripts that build param groups for MuonAdamW by hand; casing or spelling mistakes in 'kind'; copying param groups from torch.optim.AdamW-style code where 'kind' does not exist (that yields KeyError, but renamed/edited dicts yield this ValueError).","solutions":["Ensure every param group dict passed to MuonAdamW has kind='adamw' or kind='muon' (lowercase, exact).","Use the canonical group-construction code in nanochat/gpt.py (`configure_optimizers`) as the template.","Add an assertion over param_groups before training: all(g['kind'] in ('adamw','muon') for g in opt.param_groups)."],"exampleFix":"# before\nopt = MuonAdamW([dict(params=lm_head_params, lr=6e-4)])\n\n# after\nopt = MuonAdamW([dict(kind='adamw', params=lm_head_params, lr=6e-4, betas=(0.8, 0.96), eps=1e-10, weight_decay=0.01)])","handlingStrategy":"validation","validationCode":"VALID_KINDS = ('adamw', 'muon')\nfor g in param_groups:\n    assert g.get('kind') in VALID_KINDS, f\"param group kind must be one of {VALID_KINDS}, got {g.get('kind')!r}\"\nopt = MuonAdamW(param_groups)","typeGuard":"def is_valid_param_group(group: dict) -> bool:\n    return isinstance(group, dict) and group.get('kind') in ('adamw', 'muon')","tryCatchPattern":null,"preventionTips":["Build param groups via nanochat/gpt.py configure_optimizers rather than by hand.","Validate kind on every group (including later add_param_group calls) before training starts.","Use exact lowercase strings 'adamw' and 'muon'."],"tags":["nanochat","optimizer","muon","adamw","configuration"],"backgroundTag":null,"analyzedSha":"92d63d4e8bb4df75c3b71618f31ddde2378b2bcd","analyzedAt":"2026-08-15T03:11:54.371Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}