microsoft/qlib · error · ValueError

unknown metric `%s`

Error message

unknown metric `%s`

What it means

ALSTMTSModel.metric_fn only knows two metric modes: the empty string or 'loss' (negated training loss) and 'mse'. Any other value stored in self.metric makes the method fall through every branch and raise ValueError at the end of metric_fn in qlib/contrib/model/pytorch_alstm_ts.py. This metric is computed every epoch on the validation set to drive early stopping, so a bad value aborts training inside fit().

Source

Thrown at qlib/contrib/model/pytorch_alstm_ts.py:168

        if weight is None:
            weight = torch.ones_like(label)

        if self.loss == "mse":
            return self.mse(pred[mask], label[mask], weight[mask])

        raise ValueError("unknown loss `%s`" % self.loss)

    def metric_fn(self, pred, label):
        mask = torch.isfinite(label)

        if self.metric in ("", "loss"):
            return -self.loss_fn(pred[mask], label[mask])
        elif self.metric == "mse":
            mask = ~torch.isnan(label)
            weight = torch.ones_like(label)
            return -self.mse(pred[mask], label[mask], weight[mask])

        raise ValueError("unknown metric `%s`" % self.metric)

    def train_epoch(self, data_loader):
        self.ALSTM_model.train()

        for data, weight in data_loader:
            feature = data[:, :, 0:-1].to(self.device)
            label = data[:, -1, -1].to(self.device)

            pred = self.ALSTM_model(feature.float())
            loss = self.loss_fn(pred, label, weight.to(self.device))

            self.train_optimizer.zero_grad()
            loss.backward()
            torch.nn.utils.clip_grad_value_(self.ALSTM_model.parameters(), 3.0)
            self.train_optimizer.step()

    def test_epoch(self, data_loader):
        self.ALSTM_model.eval()

View on GitHub (pinned to 79633dd950)

Solutions

  1. Set metric to 'mse' or leave it as the default '' (both mean loss-based scoring) in the ALSTMTSModel constructor or workflow config.
  2. Check your workflow YAML/JSON 'metric' hyperparameter for typos or values copied from another model class.
  3. If you need a custom metric, subclass ALSTMTSModel and extend metric_fn with a new elif branch before the final raise.

Example fix

# before
model = ALSTMTSModel(metric='ic')

# after
model = ALSTMTSModel(metric='mse')
Defensive patterns

Strategy: validation

Validate before calling

from qlib.contrib.model.pytorch_alstm_ts import ALSTMTSModel
allowed = {'', 'loss', 'mse'}
assert model.metric in allowed, f"metric must be one of {allowed}, got {model.metric!r}"

Type guard

def is_valid_alstm_metric(metric: str) -> bool:
    return metric in ('', 'loss', 'mse')

Try / catch

try:
    model.fit(dataset)
except ValueError as e:
    if 'unknown metric' in str(e):
        raise ValueError(f"bad ALSTM metric {model.metric!r}; use '', 'loss' or 'mse'") from e
    raise

Prevention

When it happens

Trigger: Constructing ALSTMTSModel(metric='ic') or any string other than ''/'loss'/'mse', then calling fit(); the first validation epoch calls metric_fn(pred, label) which raises. Also triggered by typo'd YAML/JSON workflow configs that feed the metric hyperparameter verbatim.

Common situations: Copying a workflow config written for a different qlib model (e.g. LGBModel where metric='ic' is legal) and reusing it for ALSTM; upgrading qlib versions where supported metric names changed; assuming ranking metrics like IC are supported because they appear elsewhere in qlib.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/db486eb2d7997c57. Report an issue: GitHub.