{"record":{"id":"4d01d64389e21667","repo":"microsoft/qlib","slug":"unknown-metric-s-4d01d6","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_lstm.py","lineNumber":150,"sourceCode":"    def mse(self, pred, label):\n        loss = (pred - label) ** 2\n        return torch.mean(loss)\n\n    def loss_fn(self, pred, label):\n        mask = ~torch.isnan(label)\n\n        if self.loss == \"mse\":\n            return self.mse(pred[mask], label[mask])\n\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\n\n    def metric_fn(self, pred, label):\n        mask = torch.isfinite(label)\n\n        if self.metric in (\"\", \"loss\"):\n            return -self.loss_fn(pred[mask], label[mask])\n\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\n\n    def train_epoch(self, x_train, y_train):\n        x_train_values = x_train.values\n        y_train_values = np.squeeze(y_train.values)\n\n        self.lstm_model.train()\n\n        indices = np.arange(len(x_train_values))\n        np.random.shuffle(indices)\n\n        for i in range(len(indices))[:: self.batch_size]:\n            if len(indices) - i < self.batch_size:\n                break\n\n            feature = torch.from_numpy(x_train_values[indices[i : i + self.batch_size]]).float().to(self.device)\n            label = torch.from_numpy(y_train_values[indices[i : i + self.batch_size]]).float().to(self.device)\n\n            pred = self.lstm_model(feature)","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_lstm.py#L132-L168","documentation":"The LSTM model's metric_fn() supplies the validation score driving early stopping and checkpoint selection. Only self.metric in ('', 'loss') is valid and yields the negated masked MSE (higher = better). Any other string raises ValueError(\"unknown metric `%s`\") on the first validation pass inside fit().","triggerScenarios":"model.fit(...) with metric set to 'ic', 'mse', 'mae', 'acc', etc. — anything outside ('', 'loss'). Raises after the first training epoch when validation runs.","commonSituations":"Copying 'metric: ic' from Alpha158 benchmark configs used with other models; assuming the metric vocabulary is shared across all qlib contrib models.","solutions":["Use metric='' or metric='loss' in the model config.","Override metric_fn(self, pred, label) in a subclass for custom scores; mask with torch.isfinite(label) and return higher-is-better scalars.","Remember early stopping selects the maximum score, so losses must be negated."],"exampleFix":"# before\nmodel = LSTMModel(..., metric=\"ic\")\nmodel.fit(dataset)  # ValueError: unknown metric `ic`\n\n# after\nmodel = LSTMModel(..., metric=\"loss\")\nmodel.fit(dataset)","handlingStrategy":"validation","validationCode":"assert metric in (\"\", \"loss\"), \"LSTMModel supports only metric='' or 'loss'\"\nmodel = LSTMModel(..., metric=metric)","typeGuard":"def is_supported_metric(name: str) -> bool:\n    return name in (\"\", \"loss\")","tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        raise ValueError(\"metric must be '' or 'loss'\") from e\n    raise","preventionTips":["This model family scores validation by negative loss only; don't copy IC metrics over.","Custom metric_fn overrides must mask non-finite labels and return higher-is-better values.","Validate metric up front — the error otherwise appears only after the first training epoch."],"tags":["pytorch","qlib","metric","lstm","config"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}