microsoft/qlib · error · ValueError

unknown metric `%s`

Error message

unknown metric `%s`

What it means

LocalTransformerModel.metric_fn supports metric in ('', 'loss') only, both meaning 'score = negative loss' over finite labels. Any other metric string (notably 'ic', which other qlib models support) raises ValueError during validation scoring in fit().

Source

Thrown at qlib/contrib/model/pytorch_localformer.py:103

    def mse(self, pred, label):
        loss = (pred.float() - label.float()) ** 2
        return torch.mean(loss)

    def loss_fn(self, pred, label):
        mask = ~torch.isnan(label)

        if self.loss == "mse":
            return self.mse(pred[mask], label[mask])

        raise ValueError("unknown loss `%s`" % self.loss)

    def metric_fn(self, pred, label):
        mask = torch.isfinite(label)

        if self.metric in ("", "loss"):
            return -self.loss_fn(pred[mask], label[mask])

        raise ValueError("unknown metric `%s`" % self.metric)

    def train_epoch(self, x_train, y_train):
        x_train_values = x_train.values
        y_train_values = np.squeeze(y_train.values)

        self.model.train()

        indices = np.arange(len(x_train_values))
        np.random.shuffle(indices)

        for i in range(len(indices))[:: self.batch_size]:
            if len(indices) - i < self.batch_size:
                break

            feature = torch.from_numpy(x_train_values[indices[i : i + self.batch_size]]).float().to(self.device)
            label = torch.from_numpy(y_train_values[indices[i : i + self.batch_size]]).float().to(self.device)

            pred = self.model(feature)

View on GitHub (pinned to 79633dd950)

Solutions

1. Use metric='' or metric='loss'
2. Subclass and add an IC branch if IC-based early stopping is required

Example fix

# before
LocalTransformerModel(metric="ic")

# after
LocalTransformerModel(metric="loss")

assert metric in ("", "loss"), "LocalTransformerModel metric must be '' or 'loss'"

def is_supported_metric(name: str) -> bool:
    return name in ("", "loss")

When it happens

Trigger: LocalTransformerModel(metric='ic') or any value other than ''/'loss', then fit(); surfaces when the first validation score is computed.

Common situations: Defaults copied from models where 'ic' is valid; assuming every qlib torch model computes IC.

Related errors

• unknown metric `%s`
• unknown metric `%s`
• optimizer {} is not supported!
• unknown loss `%s`
• This type of input {rtype} is not supported

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/092291217d0499e5. Report an issue: GitHub.