microsoft/qlib · error · ValueError

unknown metric `%s`

Error message

unknown metric `%s`

What it means

Thrown by TCNModel.metric_fn in qlib's PyTorch TCN contribution model. During fit/predict evaluation the model converts its `metric` hyperparameter into a scoring function; only the values '' (empty string) and 'loss' (negative training loss) are implemented. Any other string reaches the terminal raise in metric_fn and aborts training with 'unknown metric `%s`'.

Source

Thrown at qlib/contrib/model/pytorch_tcn.py:162

    def mse(self, pred, label):
        loss = (pred - label) ** 2
        return torch.mean(loss)

    def loss_fn(self, pred, label):
        mask = ~torch.isnan(label)

        if self.loss == "mse":
            return self.mse(pred[mask], label[mask])

        raise ValueError("unknown loss `%s`" % self.loss)

    def metric_fn(self, pred, label):
        mask = torch.isfinite(label)

        if self.metric in ("", "loss"):
            return -self.loss_fn(pred[mask], label[mask])

        raise ValueError("unknown metric `%s`" % self.metric)

    def train_epoch(self, x_train, y_train):
        x_train_values = x_train.values
        y_train_values = np.squeeze(y_train.values)

        self.tcn_model.train()

        indices = np.arange(len(x_train_values))
        np.random.shuffle(indices)

        for i in range(len(indices))[:: self.batch_size]:
            if len(indices) - i < self.batch_size:
                break

            feature = torch.from_numpy(x_train_values[indices[i : i + self.batch_size]]).float().to(self.device)
            label = torch.from_numpy(y_train_values[indices[i : i + self.batch_size]]).float().to(self.device)

            pred = self.tcn_model(feature)

View on GitHub (pinned to 79633dd950)

Solutions

  1. Set metric='loss' (or omit it / set to '') in the TCNModel constructor — these are the only supported values.
  2. If you typo'd, check the exact spelling and casing; the comparison is against the lowercase strings '' and 'loss' only.
  3. If you need a custom metric, subclass TCNModel and override metric_fn to implement it (e.g. IC) before calling super().fit().

Example fix

# before
model = TCNModel(..., loss="mse", metric="ic")

# after
model = TCNModel(..., loss="mse", metric="loss")
Defensive patterns

Strategy: validation

Validate before calling

from qlib.contrib.model.pytorch_tcn import TCNModel
assert model_kwargs.get("metric", "") in ("", "loss"), f"TCNModel supports metric '' or 'loss', got {model_kwargs.get('metric')!r}"

Try / catch

try:
    model.fit(dataset, evals_result)
except ValueError as e:
    if "unknown metric" in str(e):
        raise ValueError(f"Fix TCNModel.metric (only '' or 'loss'): {model.metric!r}") from e
    raise

Prevention

When it happens

Trigger: Instantiating TCNModel (qlib.contrib.model.pytorch_tcn.TCNModel) and passing metric='ic', metric='auc', or any string other than '' / 'loss', then calling fit(); the first validation epoch calls metric_fn and raises.

Common situations: Users copy model configs from other qlib models (e.g. ALSTM or LightGBM workflows where metric='ic') into a TCN config. Others typo 'loss' as 'Loss' or 'mse' (mse is a valid loss value but NOT a valid metric value here).

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/c12d85adf0f77353. Report an issue: GitHub.