microsoft/qlib · error · NotImplementedError

optimizer {} is not supported!

Error message

optimizer {} is not supported!

What it means

Thrown in TransformerModel's fit setup while creating the training optimizer. The `optimizer` hyperparameter is compared case-insensitively to 'adam' and 'gd'; both branches add weight_decay=self.reg. Any other value raises NotImplementedError before any epoch runs.

Source

Thrown at qlib/contrib/model/pytorch_transformer.py:75

        self.optimizer = optimizer.lower()
        self.loss = loss
        self.n_jobs = n_jobs
        self.device = torch.device("cuda:%d" % GPU if torch.cuda.is_available() and GPU >= 0 else "cpu")
        self.seed = seed
        self.logger = get_module_logger("TransformerModel")
        self.logger.info("Naive Transformer:" "\nbatch_size : {}" "\ndevice : {}".format(self.batch_size, self.device))

        if self.seed is not None:
            np.random.seed(self.seed)
            torch.manual_seed(self.seed)

        self.model = Transformer(d_feat, d_model, nhead, num_layers, dropout, self.device)
        if optimizer.lower() == "adam":
            self.train_optimizer = optim.Adam(self.model.parameters(), lr=self.lr, weight_decay=self.reg)
        elif optimizer.lower() == "gd":
            self.train_optimizer = optim.SGD(self.model.parameters(), lr=self.lr, weight_decay=self.reg)
        else:
            raise NotImplementedError("optimizer {} is not supported!".format(optimizer))

        self.fitted = False
        self.model.to(self.device)

    @property
    def use_gpu(self):
        return self.device != torch.device("cpu")

    def mse(self, pred, label):
        loss = (pred.float() - label.float()) ** 2
        return torch.mean(loss)

    def loss_fn(self, pred, label):
        mask = ~torch.isnan(label)

        if self.loss == "mse":
            return self.mse(pred[mask], label[mask])

View on GitHub (pinned to 79633dd950)

Solutions

  1. Set optimizer='adam' or optimizer='gd' (with optional reg for weight decay).
  2. Replace 'sgd' with 'gd' — this model's spelling of plain SGD.
  3. Subclass TransformerModel and override the fit preamble to instantiate another torch.optim optimizer on self.model.parameters() if needed.

Example fix

# before
model = TransformerModel(..., optimizer="adamw", reg=1e-4)

# after
model = TransformerModel(..., optimizer="adam", reg=1e-4)
Defensive patterns

Strategy: validation

Validate before calling

optimizer = model_kwargs.get("optimizer", "adam")
assert optimizer.lower() in ("adam", "gd"), f"TransformerModel optimizer must be 'adam' or 'gd', got {optimizer!r}"

Try / catch

try:
    model.fit(dataset, evals_result)
except NotImplementedError as e:
    if "optimizer" in str(e):
        model_kwargs["optimizer"] = "adam"
        model = TransformerModel(**model_kwargs)
        model.fit(dataset, evals_result)
    else:
        raise

Prevention

When it happens

Trigger: TransformerModel(..., optimizer='sgd'|'adamw'|'rmsprop') followed by fit(); the dispatch falls through to the raise.

Common situations: Benchmark yaml configs using 'sgd'; users expecting AdamW support; typos; configs copied from models with wider optimizer support.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/ceac844ad2d3d645. Report an issue: GitHub.