{"record":{"id":"3f69955c12070d82","repo":"hiyouga/LlamaFactory","slug":"unknown-optim-training-args-optim","errorCode":null,"errorMessage":"Unknown optim: {training_args.optim}.","messagePattern":"Unknown optim: (.+?)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/train/trainer_utils.py","lineNumber":246,"sourceCode":"    for name, param in model.named_parameters():\n        if param.requires_grad:\n            trainable_params.append(param)\n            if id(param) not in id_galore_params:\n                if name in decay_param_names:\n                    decay_params.append(param)\n                else:\n                    nodecay_params.append(param)\n\n    _, optim_kwargs = Trainer.get_optimizer_cls_and_kwargs(training_args)\n\n    if training_args.optim == \"adamw_torch\":\n        optim_class = GaLoreAdamW\n    elif training_args.optim in [\"adamw_bnb_8bit\", \"adamw_8bit\", \"paged_adamw_8bit\"]:\n        optim_class = GaLoreAdamW8bit\n    elif training_args.optim == \"adafactor\":\n        optim_class = GaLoreAdafactor\n    else:\n        raise NotImplementedError(f\"Unknown optim: {training_args.optim}.\")\n\n    if finetuning_args.galore_layerwise:\n        logger.warning_rank0(\"The displayed gradient norm will be all zeros in layerwise GaLore.\")\n        if training_args.gradient_accumulation_steps != 1:\n            raise ValueError(\"Per-layer GaLore does not support gradient accumulation.\")\n\n        optimizer_dict: dict[torch.Tensor, torch.optim.Optimizer] = {}\n        for param in nodecay_params:\n            param_groups = [dict(params=[param], weight_decay=0.0)]\n            optimizer_dict[param] = optim_class(param_groups, **optim_kwargs)\n        for param in decay_params:\n            param_groups = [dict(params=[param], weight_decay=training_args.weight_decay)]\n            optimizer_dict[param] = optim_class(param_groups, **optim_kwargs)\n        for param in galore_params:  # galore params have weight decay\n            param_groups = [dict(params=[param], weight_decay=training_args.weight_decay, **galore_kwargs)]\n            optimizer_dict[param] = optim_class(param_groups, **optim_kwargs)\n\n        def optimizer_hook(param: \"torch.nn.Parameter\"):","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/train/trainer_utils.py#L228-L264","documentation":"The GaLore optimizer factory (src/llamafactory/train/trainer_utils.py:246) only maps three optimizer choices onto GaLore variants: `adamw_torch` -> GaLoreAdamW, the bitsandbytes 8-bit family (`adamw_bnb_8bit`, `adamw_8bit`, `paged_adamw_8bit`) -> GaLoreAdamW8bit, and `adafactor` -> GaLoreAdafactor. Any other `optim` value raises NotImplementedError because no GaLore equivalent exists for it.","triggerScenarios":"Config with `use_galore: true` plus an unsupported optimizer such as `adamw_torch_fused`, `adamw_anyprecision`, `lion`, or `adahessian` in TrainingArguments.","commonSituations":"Users copying a tuned optimizer setting from a non-GaLore run into a GaLore config, or defaults from a newer transformers version introducing optim names this code predates.","solutions":["Set `optim: adamw_torch` in the training YAML when using GaLore.","For memory-constrained runs use `optim: adamw_bnb_8bit` (requires bitsandbytes installed).","If you do not need GaLore, remove `use_galore: true` and keep your preferred optimizer.","For adafactor-style training use `optim: adafactor`."],"exampleFix":"# before (yaml)\nuse_galore: true\noptim: adamw_torch_fused\n\n# after\nuse_galore: true\noptim: adamw_torch","handlingStrategy":"validation","validationCode":"GALORE_OPTIMS = {\"adamw_torch\", \"adamw_bnb_8bit\", \"adamw_8bit\", \"paged_adamw_8bit\", \"adafactor\"}\nassert optim in GALORE_OPTIMS or not use_galore","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate optimizer names against the feature you enable before launching long jobs.","Prefer adamw_torch with GaLore unless memory forces 8-bit."],"tags":["galore","optimizer","sft","config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}