What causes "unknown metric `%s`" in microsoft/qlib?

Use metric='loss' (or '' to default to negative loss) in the TCNTSModel constructor.

microsoft/qlib · error · ValueError

unknown metric `%s`

Error message

unknown metric `%s`

What it means

Thrown by TCNTSModel.metric_fn, used to score each epoch for early stopping. Only '' and 'loss' (negated loss) are accepted for the `metric` hyperparameter; anything else raises before the first epoch completes.

Source

Thrown at qlib/contrib/model/pytorch_tcn_ts.py:163

    def mse(self, pred, label):
        loss = (pred - label) ** 2
        return torch.mean(loss)

    def loss_fn(self, pred, label):
        mask = ~torch.isnan(label)

        if self.loss == "mse":
            return self.mse(pred[mask], label[mask])

        raise ValueError("unknown loss `%s`" % self.loss)

    def metric_fn(self, pred, label):
        mask = torch.isfinite(label)

        if self.metric in ("", "loss"):
            return -self.loss_fn(pred[mask], label[mask])

        raise ValueError("unknown metric `%s`" % self.metric)

    def train_epoch(self, data_loader):
        self.TCN_model.train()

        for data in data_loader:
            data = torch.transpose(data, 1, 2)
            feature = data[:, 0:-1, :].to(self.device)
            label = data[:, -1, -1].to(self.device)

            pred = self.TCN_model(feature.float())
            loss = self.loss_fn(pred, label)

            self.train_optimizer.zero_grad()
            loss.backward()
            torch.nn.utils.clip_grad_value_(self.TCN_model.parameters(), 3.0)
            self.train_optimizer.step()

    def test_epoch(self, data_loader):

View on GitHub (pinned to 79633dd950)

Solutions

Use metric='loss' (or '' to default to negative loss) in the TCNTSModel constructor.
For IC-based early stopping, subclass TCNTSModel and override metric_fn with a spearman-correlation implementation over the finite-label mask.

Example fix

# before
model = TCNTSModel(..., metric="ic")

# after
model = TCNTSModel(..., metric="loss")

Defensive patterns

Strategy: validation

Validate before calling

assert model_kwargs.get("metric", "") in ("", "loss"), "TCNTSModel metric must be '' or 'loss'"

Try / catch

try:
    model.fit(ds, valid)
except ValueError as e:
    if "unknown metric" in str(e):
        model_kwargs["metric"] = "loss"
        model = TCNTSModel(**model_kwargs)
        model.fit(ds, valid)
    else:
        raise

Prevention

Don't reuse IC-style metric names with TCN time-series models.
Assert supported enum values per model class before launching long benchmark runs.

When it happens

Trigger: TCNTSModel(..., metric='ic') or any value other than ''/'loss', followed by fit(); validation calls metric_fn and hits the raise.

Common situations: Reusing a yaml/benchmark config written for models whose metric_fn supports 'ic' (e.g. some TRA/launcher workflows); typo such as 'Loss' or 'negloss'.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/0afcf051f0637ed5. Report an issue: GitHub.