{"record":{"id":"57305c0c84855e1a","repo":"microsoft/qlib","slug":"unknown-metric-s-57305c","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_localformer_ts.py","lineNumber":103,"sourceCode":"    def mse(self, pred, label):\r\n        loss = (pred.float() - label.float()) ** 2\r\n        return torch.mean(loss)\r\n\r\n    def loss_fn(self, pred, label):\r\n        mask = ~torch.isnan(label)\r\n\r\n        if self.loss == \"mse\":\r\n            return self.mse(pred[mask], label[mask])\r\n\r\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\r\n\r\n    def metric_fn(self, pred, label):\r\n        mask = torch.isfinite(label)\r\n\r\n        if self.metric in (\"\", \"loss\"):\r\n            return -self.loss_fn(pred[mask], label[mask])\r\n\r\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\r\n\r\n    def train_epoch(self, data_loader):\r\n        self.model.train()\r\n\r\n        for data in data_loader:\r\n            feature = data[:, :, 0:-1].to(self.device)\r\n            label = data[:, -1, -1].to(self.device)\r\n\r\n            pred = self.model(feature.float())  # .float()\r\n            loss = self.loss_fn(pred, label)\r\n\r\n            self.train_optimizer.zero_grad()\r\n            loss.backward()\r\n            torch.nn.utils.clip_grad_value_(self.model.parameters(), 3.0)\r\n            self.train_optimizer.step()\r\n\r\n    def test_epoch(self, data_loader):\r\n        self.model.eval()\r","sourceCodeStart":85,"sourceCodeEnd":121,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_localformer_ts.py#L85-L121","documentation":"LOCALTransformerModel.metric_fn() computes the validation score used for early stopping and best-epoch tracking. Only self.metric in ('', 'loss') is supported: it returns the negated masked loss (higher is better). Any other metric string raises ValueError(\"unknown metric `%s`\") during validation in fit(), not at construction time.","triggerScenarios":"Calling model.fit(...) with metric set to anything besides '' or 'loss' — e.g. 'ic', 'mse', 'accuracy'. The error is raised on the first validation pass after the first training epoch.","commonSituations":"Porting 'ic'-style metric configs from other qlib models (e.g. pytorch_nn or GBDT workflows where IC is common); assuming standard sklearn metric names work; leaving a metric from a copied YAML that this model family does not implement.","solutions":["Set metric='' or metric='loss' (both mean: use negative MSE as the score) in the model kwargs.","To use a custom metric, subclass and override metric_fn(self, pred, label); return a scalar where higher is better, and mask non-finite labels with torch.isfinite(label).","Note the sign convention: scores are maximized (best_score starts at -np.inf), so return negative values for losses."],"exampleFix":"# before\nmodel = LOCALTransformerModel(..., metric=\"ic\")\nmodel.fit(dataset)  # ValueError: unknown metric `ic` at validation\n\n# after\nmodel = LOCALTransformerModel(..., metric=\"loss\")\nmodel.fit(dataset)","handlingStrategy":"validation","validationCode":"metric = \"loss\"\nassert metric in (\"\", \"loss\"), \"LOCALTransformerModel supports only metric='' or 'loss'\"\nmodel = LOCALTransformerModel(..., metric=metric)","typeGuard":"def is_supported_metric(name: str) -> bool:\n    return isinstance(name, str) and name in (\"\", \"loss\")","tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        raise ValueError(\"Set metric='' or 'loss' (negative MSE as score)\") from e\n    raise","preventionTips":["Do not port 'ic'/'rank_ic' metrics from other models into this model's config.","Early stopping maximizes the returned score — custom metrics must be higher-is-better.","Validate metric early since the error otherwise surfaces only at first validation."],"tags":["pytorch","qlib","metric","transformer","config"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}