{"record":{"id":"92c36ad4ed6baef4","repo":"microsoft/qlib","slug":"unknown-metric-s-92c36a","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_gats.py","lineNumber":162,"sourceCode":"    def mse(self, pred, label):\n        loss = (pred - label) ** 2\n        return torch.mean(loss)\n\n    def loss_fn(self, pred, label):\n        mask = ~torch.isnan(label)\n\n        if self.loss == \"mse\":\n            return self.mse(pred[mask], label[mask])\n\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\n\n    def metric_fn(self, pred, label):\n        mask = torch.isfinite(label)\n\n        if self.metric in (\"\", \"loss\"):\n            return -self.loss_fn(pred[mask], label[mask])\n\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\n\n    def get_daily_inter(self, df, shuffle=False):\n        # organize the train data into daily batches\n        daily_count = df.groupby(level=0, group_keys=False).size().values\n        daily_index = np.roll(np.cumsum(daily_count), 1)\n        daily_index[0] = 0\n        if shuffle:\n            # shuffle data\n            daily_shuffle = list(zip(daily_index, daily_count))\n            np.random.shuffle(daily_shuffle)\n            daily_index, daily_count = zip(*daily_shuffle)\n        return daily_index, daily_count\n\n    def train_epoch(self, x_train, y_train):\n        x_train_values = x_train.values\n        y_train_values = np.squeeze(y_train.values)\n        self.GAT_model.train()\n","sourceCodeStart":144,"sourceCodeEnd":180,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_gats.py#L144-L180","documentation":"GATsModel.metric_fn accepts only the empty string or 'loss' as self.metric, returning the negated training loss as the early-stopping score. Unlike ALSTM it has no 'mse' branch, so metric='mse' is also rejected. Any unmatched value raises ValueError on the first validation pass inside fit().","triggerScenarios":"Constructing GATsModel(metric='mse') or metric='ic' and calling fit(); the first validation epoch calls metric_fn which raises.","commonSituations":"Assuming the metric options are identical across qlib PyTorch models (ALSTM supports 'mse', GATs does not); copy-pasting model hyperparameter blocks between workflow configs.","solutions":["Set metric='' or 'loss' (both mean loss-based scoring) in the GATsModel config.","Remove metric='mse' from configs ported from ALSTM workflows.","Subclass GATsModel and add metric branches in metric_fn if you need another metric."],"exampleFix":"# before\nmodel = GATsModel(metric='mse')\n\n# after\nmodel = GATsModel(metric='loss')","handlingStrategy":"validation","validationCode":"assert model.metric in ('', 'loss'), f\"GATsModel metric must be '' or 'loss', got {model.metric!r}\"","typeGuard":"def is_supported_gats_metric(metric: str) -> bool:\n    return metric in ('', 'loss')","tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if 'unknown metric' in str(e):\n        model.metric = 'loss'\n        model.fit(dataset)\n    else:\n        raise","preventionTips":["Note GATs supports fewer metrics than ALSTM (no 'mse' branch).","Validate metric per-model, not globally across your experiment suite.","Keep a per-model schema of supported hyperparameters next to your workflow configs."],"tags":["pytorch","qlib","metrics","config-validation","gats"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}