microsoft/qlib · error · ValueError
unknown metric `%s`
Error message
unknown metric `%s`
What it means
Raised by DNNModelPytorch.metric_fn when the `metric` hyper-parameter is neither "" nor "loss". The metric used for early-stopping/validation scoring is hard-coded: only the (negated implicit) training loss is available via those two aliases. Any other string ("ic", "auc", ...) falls through to the raise during validation of the first epoch.
Source
Thrown at qlib/contrib/model/pytorch_general_nn.py:172
def loss_fn(self, pred, label, weight=None):
mask = ~torch.isnan(label)
if weight is None:
weight = torch.ones_like(label)
if self.loss == "mse":
return self.mse(pred[mask], label[mask].view(-1, 1), weight[mask])
raise ValueError("unknown loss `%s`" % self.loss)
def metric_fn(self, pred, label):
mask = torch.isfinite(label)
if self.metric in ("", "loss"):
return self.loss_fn(pred[mask], label[mask])
raise ValueError("unknown metric `%s`" % self.metric)
def _get_fl(self, data: torch.Tensor):
"""
get feature and label from data
- Handle the different data shape of time series and tabular data
Parameters
----------
data : torch.Tensor
input data which maybe 3 dimension or 2 dimension
- 3dim: [batch_size, time_step, feature_dim]
- 2dim: [batch_size, feature_dim]
Returns
-------
Tuple[torch.Tensor, torch.Tensor]
"""
if data.dim() == 3:View on GitHub (pinned to 79633dd950)
Solutions
- Set metric to "" or "loss" in the model parameters (validation score becomes the loss).
- If you need "ic" or another metric, subclass DNNModelPytorch and extend metric_fn with your branch before the raise.
- Verify the YAML key is under the correct model handler so the value actually reaches the constructor.
Example fix
# before DNNModelPytorch(metric="ic", ...) # ValueError: unknown metric `ic` # after DNNModelPytorch(metric="loss", ...)
Defensive patterns
Strategy: validation
Validate before calling
allowed = {"", "loss"}
assert params.get("metric", "") in allowed, f"metric must be one of {allowed} for DNNModelPytorch" Type guard
def is_supported_metric(metric: str) -> bool:
return isinstance(metric, str) and metric in {"", "loss"} Try / catch
try:
model.fit(dataset)
except ValueError as e:
if "unknown metric" in str(e):
# fall back to loss-based scoring
params["metric"] = "loss"
model = DNNModelPytorch(**params)
model.fit(dataset)
else:
raise Prevention
- Validate metric names per model class before launching long training jobs.
- Prefer leaving metric unset (defaults to loss scoring) unless you know the model supports a named metric.
When it happens
Trigger: Constructing DNNModelPytorch(metric="ic") and calling fit() with a validation segment; metric_fn is called per validation batch and raises on the first one. Note also the mask here uses torch.isfinite on the label and then passes pred[mask], label[mask] into loss_fn, so shape mismatches can occur before this raise if labels contain non-finite values.
Common situations: Porting a workflow config from GRU/LSTM models where metric="ic" is valid; assuming the generic-sounding `metric` arg accepts standard qlib metrics like "ic" or "icir"; leaving metric unset in one model but copying "ic" from a benchmark config.
Related errors
AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15).
Data as JSON: /api/errors/519ab9f83913cdb9.
Report an issue: GitHub.