microsoft/qlib · error · ValueError
unknown metric `%s`
Error message
unknown metric `%s`
What it means
Raised by SANDWICH model metric_fn in qlib/contrib/model/pytorch_sandwich.py:249 when self.metric is not '' or 'loss'. Only two tokens are accepted: empty string and 'loss', both meaning 'use negative training loss as the early-stopping metric'. Any other metric name (e.g. 'ic') raises ValueError because no other metric is implemented.
Source
Thrown at qlib/contrib/model/pytorch_sandwich.py:249
def mse(self, pred, label):
loss = (pred - label) ** 2
return torch.mean(loss)
def loss_fn(self, pred, label):
mask = ~torch.isnan(label)
if self.loss == "mse":
return self.mse(pred[mask], label[mask])
raise ValueError("unknown loss `%s`" % self.loss)
def metric_fn(self, pred, label):
mask = torch.isfinite(label)
if self.metric in ("", "loss"):
return -self.loss_fn(pred[mask], label[mask])
raise ValueError("unknown metric `%s`" % self.metric)
def train_epoch(self, x_train, y_train):
x_train_values = x_train.values
y_train_values = np.squeeze(y_train.values)
self.sandwich_model.train()
indices = np.arange(len(x_train_values))
np.random.shuffle(indices)
for i in range(len(indices))[:: self.batch_size]:
if len(indices) - i < self.batch_size:
break
feature = torch.from_numpy(x_train_values[indices[i : i + self.batch_size]]).float().to(self.device)
label = torch.from_numpy(y_train_values[indices[i : i + self.batch_size]]).float().to(self.device)
pred = self.sandwich_model(feature)
loss = self.loss_fn(pred, label)View on GitHub (pinned to 79633dd950)
Solutions
- Set metric: '' (or omit it) or metric: 'loss' in the model kwargs.
- If you need IC-based early stopping, subclass and override metric_fn() with an IC computation on masked labels.
Example fix
# before kwargs: metric: ic # after kwargs: metric: loss # or '' (default: negative training loss)
Defensive patterns
Strategy: validation
Validate before calling
metric = config.get("metric", "")
assert metric in ("", "loss"), f"metric must be '' or 'loss', got {metric!r}" Type guard
def is_supported_sandwich_metric(metric: str) -> bool:
return metric in ("", "loss") Try / catch
try:
model.fit(dataset)
except ValueError as e:
if "unknown metric" in str(e):
raise ValueError("Set metric='' or 'loss'; override metric_fn() for custom metrics") from e
raise Prevention
- Default the metric kwarg to '' in generated configs so the guard never triggers.
- Remember only the negative training loss is available for early stopping in this model.
When it happens
Trigger: Passing metric='ic', metric='auc', or any non-empty string other than 'loss' to the sandwich model constructor and calling fit(); the raise fires on the first validation evaluation inside the training loop.
Common situations: Copying metric: 'ic' from a DNNModelPytorch/qlib workflow config into the sandwich model; assuming custom metric names from qlib's signal analysis are accepted here.
Related errors
AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15).
Data as JSON: /api/errors/0f0df4a84999ec6d.
Report an issue: GitHub.