microsoft/qlib · error · ValueError

unknown metric `%s`

Error message

unknown metric `%s`

What it means

GATsTSModel.metric_fn validates self.metric against only '' or 'loss'; there is no 'mse' branch, so even metric='mse' raises ValueError during the first validation pass in fit(). Early stopping depends on this metric, so training aborts before any epoch completes.

Source

Thrown at qlib/contrib/model/pytorch_gats_ts.py:182

    def mse(self, pred, label):
        loss = (pred - label) ** 2
        return torch.mean(loss)

    def loss_fn(self, pred, label):
        mask = ~torch.isnan(label)

        if self.loss == "mse":
            return self.mse(pred[mask], label[mask])

        raise ValueError("unknown loss `%s`" % self.loss)

    def metric_fn(self, pred, label):
        mask = torch.isfinite(label)

        if self.metric in ("", "loss"):
            return -self.loss_fn(pred[mask], label[mask])

        raise ValueError("unknown metric `%s`" % self.metric)

    def get_daily_inter(self, df, shuffle=False):
        # organize the train data into daily batches
        daily_count = df.groupby(level=0, group_keys=False).size().values
        daily_index = np.roll(np.cumsum(daily_count), 1)
        daily_index[0] = 0
        if shuffle:
            # shuffle data
            daily_shuffle = list(zip(daily_index, daily_count))
            np.random.shuffle(daily_shuffle)
            daily_index, daily_count = zip(*daily_shuffle)
        return daily_index, daily_count

    def train_epoch(self, data_loader):
        self.GAT_model.train()

        for data in data_loader:
            data = data.squeeze()

View on GitHub (pinned to 79633dd950)

Solutions

  1. Set metric='' or 'loss'.
  2. Strip metric='mse' from configs ported from ALSTM-based workflows.
  3. Subclass GATsTSModel to extend metric_fn if a custom metric is required.

Example fix

# before
model = GATsTSModel(metric='mse')

# after
model = GATsTSModel(metric='loss')
Defensive patterns

Strategy: validation

Validate before calling

assert model.metric in ('', 'loss'), f"GATsTSModel metric must be '' or 'loss', got {model.metric!r}"

Type guard

def is_supported_gats_metric(metric: str) -> bool:
    return metric in ('', 'loss')

Try / catch

try:
    model.fit(dataset)
except ValueError as e:
    if 'unknown metric' in str(e):
        model.metric = 'loss'
        model.fit(dataset)
    else:
        raise

Prevention

When it happens

Trigger: GATsTSModel(metric='mse') or metric='ic' followed by fit(); first validation epoch calls metric_fn and raises.

Common situations: Reusing metric settings from ALSTM (which supports 'mse'); assuming a shared metric vocabulary across qlib contrib models; config templates with non-empty defaults.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/c0effc8035a4f1b1. Report an issue: GitHub.