{"record":{"id":"7455374fe11220e3","repo":"microsoft/qlib","slug":"unknown-metric-s-745537","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_transformer.py","lineNumber":102,"sourceCode":"    def mse(self, pred, label):\r\n        loss = (pred.float() - label.float()) ** 2\r\n        return torch.mean(loss)\r\n\r\n    def loss_fn(self, pred, label):\r\n        mask = ~torch.isnan(label)\r\n\r\n        if self.loss == \"mse\":\r\n            return self.mse(pred[mask], label[mask])\r\n\r\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\r\n\r\n    def metric_fn(self, pred, label):\r\n        mask = torch.isfinite(label)\r\n\r\n        if self.metric in (\"\", \"loss\"):\r\n            return -self.loss_fn(pred[mask], label[mask])\r\n\r\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\r\n\r\n    def train_epoch(self, x_train, y_train):\r\n        x_train_values = x_train.values\r\n        y_train_values = np.squeeze(y_train.values)\r\n\r\n        self.model.train()\r\n\r\n        indices = np.arange(len(x_train_values))\r\n        np.random.shuffle(indices)\r\n\r\n        for i in range(len(indices))[:: self.batch_size]:\r\n            if len(indices) - i < self.batch_size:\r\n                break\r\n\r\n            feature = torch.from_numpy(x_train_values[indices[i : i + self.batch_size]]).float().to(self.device)\r\n            label = torch.from_numpy(y_train_values[indices[i : i + self.batch_size]]).float().to(self.device)\r\n\r\n            pred = self.model(feature)\r","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_transformer.py#L84-L120","documentation":"Thrown by TransformerModel.metric_fn, which scores each epoch for early stopping. Supported metric values are '' and 'loss' (negative MSE on the finite-label mask); anything else raises the first time validation runs.","triggerScenarios":"TransformerModel(..., metric='ic'|'auc'|anything) followed by fit(); the validation pass calls metric_fn and hits the raise.","commonSituations":"Sharing one hyperparameter dict across several qlib models where 'ic' is valid elsewhere; typos like 'Loss'.","solutions":["Use metric='loss' or '' in the TransformerModel constructor.","Keep per-model hyperparameter dicts instead of one shared dict so unsupported metric names don't leak between models.","Subclass and override metric_fn (e.g. IC on the finite mask) if a different early-stopping criterion is needed."],"exampleFix":"# before\nshared_kwargs = {\"loss\": \"mse\", \"metric\": \"ic\"}\nmodel = TransformerModel(**shared_kwargs)\n\n# after\nmodel = TransformerModel(..., loss=\"mse\", metric=\"loss\")","handlingStrategy":"validation","validationCode":"assert model_kwargs.get(\"metric\", \"\") in (\"\", \"loss\"), \"TransformerModel metric must be '' or 'loss'\"","typeGuard":null,"tryCatchPattern":"try:\n    model.fit(dataset, evals_result)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        model_kwargs[\"metric\"] = \"loss\"\n        model = TransformerModel(**model_kwargs)\n        model.fit(dataset, evals_result)\n    else:\n        raise","preventionTips":["Never share one metric value across different qlib model classes.","Strip whitespace from config strings before passing them to model constructors."],"tags":["qlib","pytorch","transformer","hyperparameter","validation"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}