{"record":{"id":"0f0df4a84999ec6d","repo":"microsoft/qlib","slug":"unknown-metric-s-0f0df4","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_sandwich.py","lineNumber":249,"sourceCode":"    def mse(self, pred, label):\n        loss = (pred - label) ** 2\n        return torch.mean(loss)\n\n    def loss_fn(self, pred, label):\n        mask = ~torch.isnan(label)\n\n        if self.loss == \"mse\":\n            return self.mse(pred[mask], label[mask])\n\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\n\n    def metric_fn(self, pred, label):\n        mask = torch.isfinite(label)\n\n        if self.metric in (\"\", \"loss\"):\n            return -self.loss_fn(pred[mask], label[mask])\n\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\n\n    def train_epoch(self, x_train, y_train):\n        x_train_values = x_train.values\n        y_train_values = np.squeeze(y_train.values)\n        self.sandwich_model.train()\n\n        indices = np.arange(len(x_train_values))\n        np.random.shuffle(indices)\n\n        for i in range(len(indices))[:: self.batch_size]:\n            if len(indices) - i < self.batch_size:\n                break\n\n            feature = torch.from_numpy(x_train_values[indices[i : i + self.batch_size]]).float().to(self.device)\n            label = torch.from_numpy(y_train_values[indices[i : i + self.batch_size]]).float().to(self.device)\n\n            pred = self.sandwich_model(feature)\n            loss = self.loss_fn(pred, label)","sourceCodeStart":231,"sourceCodeEnd":267,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_sandwich.py#L231-L267","documentation":"Raised by SANDWICH model metric_fn in qlib/contrib/model/pytorch_sandwich.py:249 when self.metric is not '' or 'loss'. Only two tokens are accepted: empty string and 'loss', both meaning 'use negative training loss as the early-stopping metric'. Any other metric name (e.g. 'ic') raises ValueError because no other metric is implemented.","triggerScenarios":"Passing metric='ic', metric='auc', or any non-empty string other than 'loss' to the sandwich model constructor and calling fit(); the raise fires on the first validation evaluation inside the training loop.","commonSituations":"Copying metric: 'ic' from a DNNModelPytorch/qlib workflow config into the sandwich model; assuming custom metric names from qlib's signal analysis are accepted here.","solutions":["Set metric: '' (or omit it) or metric: 'loss' in the model kwargs.","If you need IC-based early stopping, subclass and override metric_fn() with an IC computation on masked labels."],"exampleFix":"# before\nkwargs:\n  metric: ic\n\n# after\nkwargs:\n  metric: loss   # or '' (default: negative training loss)","handlingStrategy":"validation","validationCode":"metric = config.get(\"metric\", \"\")\nassert metric in (\"\", \"loss\"), f\"metric must be '' or 'loss', got {metric!r}\"","typeGuard":"def is_supported_sandwich_metric(metric: str) -> bool:\n    return metric in (\"\", \"loss\")","tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        raise ValueError(\"Set metric='' or 'loss'; override metric_fn() for custom metrics\") from e\n    raise","preventionTips":["Default the metric kwarg to '' in generated configs so the guard never triggers.","Remember only the negative training loss is available for early stopping in this model."],"tags":["qlib","pytorch","metric","config","early-stopping","sandwich"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}