microsoft/qlib · error · ValueError
unknown metric `%s`
Error message
unknown metric `%s`
What it means
Raised by SFM model metric_fn in qlib/contrib/model/pytorch_sfm.py:434 when self.metric is neither '' nor 'loss'. The only implemented early-stopping metric is the negative training loss; passing 'ic', 'auc', etc. raises ValueError at the first validation scoring inside fit().
Source
Thrown at qlib/contrib/model/pytorch_sfm.py:434
def mse(self, pred, label):
loss = (pred - label) ** 2
return torch.mean(loss)
def loss_fn(self, pred, label):
mask = ~torch.isnan(label)
if self.loss == "mse":
return self.mse(pred[mask], label[mask])
raise ValueError("unknown loss `%s`" % self.loss)
def metric_fn(self, pred, label):
mask = torch.isfinite(label)
if self.metric in ("", "loss"):
return -self.loss_fn(pred[mask], label[mask])
raise ValueError("unknown metric `%s`" % self.metric)
def predict(self, dataset: DatasetH, segment: Union[Text, slice] = "test"):
if not self.fitted:
raise ValueError("model is not fitted yet!")
x_test = dataset.prepare(segment, col_set="feature", data_key=DataHandlerLP.DK_I)
index = x_test.index
self.sfm_model.eval()
x_values = x_test.values
sample_num = x_values.shape[0]
preds = []
for begin in range(sample_num)[:: self.batch_size]:
if sample_num - begin < self.batch_size:
end = sample_num
else:
end = begin + self.batch_size
View on GitHub (pinned to 79633dd950)
Solutions
- Use metric: '' (default) or metric: 'loss'.
- Subclass and override metric_fn() to implement IC or another custom early-stopping metric.
Example fix
# before kwargs: metric: ic # after kwargs: metric: "" # or 'loss'
Defensive patterns
Strategy: validation
Validate before calling
metric = config.get("metric", "")
assert metric in ("", "loss"), f"SFM metric must be '' or 'loss', got {metric!r}" Type guard
def is_supported_sfm_metric(metric: str) -> bool:
return metric in ("", "loss") Try / catch
try:
model.fit(dataset)
except ValueError as e:
if "unknown metric" in str(e):
raise ValueError("SFM early stopping only tracks the loss; set metric='' or 'loss'") from e
raise Prevention
- Leave metric unset in configs unless you know the model's supported set.
- Read the model's metric_fn source when porting configs between models.
When it happens
Trigger: Passing metric='ic' or any unsupported token in SFM kwargs; the raise occurs inside the fit() evaluation loop, after training has already started for the epoch.
Common situations: Workflow configs ported from DNNModel-based examples that use IC; users assuming qlib's analysis metrics are available as training metrics.
Related errors
AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15).
Data as JSON: /api/errors/330c427da5ca9026.
Report an issue: GitHub.