{"record":{"id":"dbf6b688545de650","repo":"microsoft/qlib","slug":"unknown-metric-s-dbf6b6","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_lstm_ts.py","lineNumber":158,"sourceCode":"\n    def loss_fn(self, pred, label, weight):\n        mask = ~torch.isnan(label)\n\n        if weight is None:\n            weight = torch.ones_like(label)\n\n        if self.loss == \"mse\":\n            return self.mse(pred[mask], label[mask], weight[mask])\n\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\n\n    def metric_fn(self, pred, label):\n        mask = torch.isfinite(label)\n\n        if self.metric in (\"\", \"loss\"):\n            return -self.loss_fn(pred[mask], label[mask], weight=None)\n\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\n\n    def train_epoch(self, data_loader):\n        self.LSTM_model.train()\n\n        for data, weight in data_loader:\n            feature = data[:, :, 0:-1].to(self.device)\n            label = data[:, -1, -1].to(self.device)\n\n            pred = self.LSTM_model(feature.float())\n            loss = self.loss_fn(pred, label, weight.to(self.device))\n\n            self.train_optimizer.zero_grad()\n            loss.backward()\n            torch.nn.utils.clip_grad_value_(self.LSTM_model.parameters(), 3.0)\n            self.train_optimizer.step()\n\n    def test_epoch(self, data_loader):\n        self.LSTM_model.eval()","sourceCodeStart":140,"sourceCodeEnd":176,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_lstm_ts.py#L140-L176","documentation":"The TS LSTM metric_fn() only supports self.metric in ('', 'loss'), returning -loss_fn (with weight=None, i.e. unweighted) as the higher-is-better validation score. Any other metric string raises ValueError(\"unknown metric `%s`\") during the first validation pass in fit(), after the first epoch of training has already run.","triggerScenarios":"model.fit(...) with metric set to any string other than '' or 'loss' — 'ic', 'rank_ic', 'mse', etc.","commonSituations":"Benchmark configs written for models that accept IC-style metrics; users assuming the metric kwarg mirrors qlib's signal analysis metrics.","solutions":["Set metric='' or metric='loss'.","Override metric_fn(self, pred, label) in a subclass for custom scores; note the existing signature passes weight=None through loss_fn, so custom metrics should mask non-finite labels themselves.","Keep the higher-is-better convention so early stopping and best-epoch logic behave correctly."],"exampleFix":"# before\nmodel = LSTMModel(..., metric=\"rank_ic\")\nmodel.fit(dataset)  # ValueError: unknown metric `rank_ic`\n\n# after\nmodel = LSTMModel(..., metric=\"loss\")\nmodel.fit(dataset)","handlingStrategy":"validation","validationCode":"assert metric in (\"\", \"loss\"), \"TS LSTM supports only metric='' or 'loss'\"\nmodel = LSTMModel(..., metric=metric)","typeGuard":"def is_supported_metric(name: str) -> bool:\n    return name in (\"\", \"loss\")","tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        raise ValueError(\"metric must be '' or 'loss'\") from e\n    raise","preventionTips":["Validation scoring here is unweighted negative loss (weight=None); IC metrics need a custom metric_fn.","Validate metric early to avoid losing the first epoch's compute.","Keep the higher-is-better convention for any custom metric."],"tags":["pytorch","qlib","metric","lstm","config"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}