{"record":{"id":"071930cff50ba8ed","repo":"microsoft/qlib","slug":"unknown-metric-s-071930","errorCode":null,"errorMessage":"unknown metric `%s`","messagePattern":"unknown metric `(.+?)`","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_igmtf.py","lineNumber":169,"sourceCode":"            return self.mse(pred[mask], label[mask])\n\n        raise ValueError(\"unknown loss `%s`\" % self.loss)\n\n    def metric_fn(self, pred, label):\n        mask = torch.isfinite(label)\n\n        if self.metric == \"ic\":\n            x = pred[mask]\n            y = label[mask]\n\n            vx = x - torch.mean(x)\n            vy = y - torch.mean(y)\n            return torch.sum(vx * vy) / (torch.sqrt(torch.sum(vx**2)) * torch.sqrt(torch.sum(vy**2)))\n\n        if self.metric == (\"\", \"loss\"):\n            return -self.loss_fn(pred[mask], label[mask])\n\n        raise ValueError(\"unknown metric `%s`\" % self.metric)\n\n    def get_daily_inter(self, df, shuffle=False):\n        # organize the train data into daily batches\n        daily_count = df.groupby(level=0, group_keys=False).size().values\n        daily_index = np.roll(np.cumsum(daily_count), 1)\n        daily_index[0] = 0\n        if shuffle:\n            # shuffle data\n            daily_shuffle = list(zip(daily_index, daily_count))\n            np.random.shuffle(daily_shuffle)\n            daily_index, daily_count = zip(*daily_shuffle)\n        return daily_index, daily_count\n\n    def get_train_hidden(self, x_train):\n        x_train_values = x_train.values\n        daily_index, daily_count = self.get_daily_inter(x_train, shuffle=True)\n        self.igmtf_model.eval()\n        train_hidden = []","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_igmtf.py#L151-L187","documentation":"IGMTFModel.metric_fn supports metric='ic' (Pearson correlation of prediction vs label over finite labels) and metric ('', 'loss') which is intended to mean 'use negative loss as the score'. Note the second check compares with == against a tuple, so the strings '' and 'loss' never actually match — a known qlib bug — meaning anything other than 'ic' always raises.","triggerScenarios":"Calling fit() with self.metric set to anything except 'ic'. Due to the tuple-comparison bug, even metric='' or metric='loss' (the values other qlib models accept) raise this error in IGMTFModel.","commonSituations":"Using hyperparameter defaults copied from pytorch_gru/ALSTM workflows where metric='' is common; expecting '' to mean 'loss' as in other qlib models and hitting the buggy tuple comparison.","solutions":["Set metric='ic' for IGMTFModel","If you want loss-as-metric, patch metric_fn locally to use `if self.metric in ('', 'loss')` (this is the upstream bug) or subclass and override metric_fn"],"exampleFix":"# before\nIGMTFModel(metric=\"loss\")  # raises: code compares self.metric == (\"\", \"loss\")\n\n# after\nIGMTFModel(metric=\"ic\")\n# or patch upstream bug:\n#   if self.metric in (\"\", \"loss\"): return -self.loss_fn(pred[mask], label[mask])","handlingStrategy":"validation","validationCode":"assert metric == \"ic\", \"IGMTFModel metric_fn only works with 'ic' (the ('', 'loss') branch is buggy upstream)\"","typeGuard":"def is_supported_igmtf_metric(name):\n    return name == \"ic\"","tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"unknown metric\" in str(e):\n        model.metric = \"ic\"\n        model.fit(dataset)\n    else:\n        raise","preventionTips":["Use metric='ic' for IGMTFModel; do not copy ''/'loss' defaults from other models","Patch or override the buggy `== ('', 'loss')` tuple comparison if you need loss-as-metric"],"tags":["qlib","igmtf","metric","invalid-argument","upstream-bug"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}