{"record":{"id":"a124182c512145e5","repo":"microsoft/qlib","slug":"unknown-metric-s-a12418","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_transformer_ts.py","lineNumber":100,"sourceCode":"    def mse(self, pred, label):\r\n        loss = (pred.float() - label.float()) ** 2\r\n        return torch.mean(loss)\r\n\r\n    def loss_fn(self, pred, label):\r\n        mask = ~torch.isnan(label)\r\n\r\n        if self.loss == \"mse\":\r\n            return self.mse(pred[mask], label[mask])\r\n\r\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\r\n\r\n    def metric_fn(self, pred, label):\r\n        mask = torch.isfinite(label)\r\n\r\n        if self.metric in (\"\", \"loss\"):\r\n            return -self.loss_fn(pred[mask], label[mask])\r\n\r\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\r\n\r\n    def train_epoch(self, data_loader):\r\n        self.model.train()\r\n\r\n        for data in data_loader:\r\n            feature = data[:, :, 0:-1].to(self.device)\r\n            label = data[:, -1, -1].to(self.device)\r\n\r\n            pred = self.model(feature.float())  # .float()\r\n            loss = self.loss_fn(pred, label)\r\n\r\n            self.train_optimizer.zero_grad()\r\n            loss.backward()\r\n            torch.nn.utils.clip_grad_value_(self.model.parameters(), 3.0)\r\n            self.train_optimizer.step()\r\n\r\n    def test_epoch(self, data_loader):\r\n        self.model.eval()\r","sourceCodeStart":82,"sourceCodeEnd":118,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_transformer_ts.py#L82-L118","documentation":"Thrown by TransformerTSModel.metric_fn, the per-epoch scoring function used for early stopping and best-model selection. Only '' and 'loss' (negated training loss) are implemented; any other `metric` string raises during the first validation pass.","triggerScenarios":"TransformerTSModel(..., metric='ic'|'loss '|any non-empty value other than 'loss') then fit(); metric_fn hits the raise.","commonSituations":"Configs copied from IC-based workflows; trailing whitespace in the metric string (e.g. 'loss ') which fails the equality; shared hyperparameter dicts across heterogeneous models.","solutions":["Set metric='loss' or '' exactly (no extra whitespace) in the TransformerTSModel constructor.","Use per-model config dicts so metric names valid in other models don't leak into this one.","Subclass and override metric_fn for a custom early-stopping metric like IC."],"exampleFix":"# before\nmodel = TransformerTSModel(..., metric=\"ic\")\n\n# after\nmodel = TransformerTSModel(..., metric=\"loss\")","handlingStrategy":"validation","validationCode":"metric = model_kwargs.get(\"metric\", \"\")\nassert metric in (\"\", \"loss\"), f\"TransformerTSModel metric must be '' or 'loss', got {metric!r}\"","typeGuard":null,"tryCatchPattern":"try:\n    model.fit(ds, valid)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        model_kwargs[\"metric\"] = \"loss\"\n        model = TransformerTSModel(**model_kwargs)\n        model.fit(ds, valid)\n    else:\n        raise","preventionTips":["Strip/normalize metric strings from configs to avoid whitespace/case mismatches.","Use metric='loss' uniformly for loss-based early stopping across these models."],"tags":["qlib","pytorch","transformer","hyperparameter","validation"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}