microsoft/qlib · error · ValueError

unknown metric `%s`

Error message

unknown metric `%s`

What it means

Raised by TabNet model metric_fn in qlib/contrib/model/pytorch_tabnet.py:378 when self.metric is neither '' nor 'loss'. Early stopping in this model can only track negative masked MSE ('loss' or empty string); passing 'ic', 'auc', etc. raises ValueError on the first validation scoring in fit().

Source

Thrown at qlib/contrib/model/pytorch_tabnet.py:378

        """
        Pretrain loss function defined in the original paper, read "Tabular self-supervised learning" in https://arxiv.org/pdf/1908.07442.pdf
        """
        down_mean = torch.mean(f, dim=0)
        down = torch.sqrt(torch.sum(torch.square(f - down_mean), dim=0))
        up = (f_hat - f) * S
        return torch.sum(torch.square(up / down))

    def loss_fn(self, pred, label):
        mask = ~torch.isnan(label)
        if self.loss == "mse":
            return self.mse(pred[mask], label[mask])
        raise ValueError("unknown loss `%s`" % self.loss)

    def metric_fn(self, pred, label):
        mask = torch.isfinite(label)
        if self.metric in ("", "loss"):
            return -self.loss_fn(pred[mask], label[mask])
        raise ValueError("unknown metric `%s`" % self.metric)

    def mse(self, pred, label):
        loss = (pred - label) ** 2
        return torch.mean(loss)


class FinetuneModel(nn.Module):
    """
    FinuetuneModel for adding a layer by the end
    """

    def __init__(self, input_dim, output_dim, trained_model):
        super().__init__()
        self.model = trained_model
        self.fc = nn.Linear(input_dim, output_dim)

    def forward(self, x, priors):
        return self.fc(self.model(x, priors)[0]).squeeze()  # take the vec out

View on GitHub (pinned to 79633dd950)

Solutions

  1. Set metric: '' or metric: 'loss'.
  2. Subclass and override metric_fn() with a custom metric (e.g. IC) for alternative early stopping.

Example fix

# before
kwargs:
  metric: ic

# after
kwargs:
  metric: loss   # or ''
Defensive patterns

Strategy: validation

Validate before calling

metric = config.get("metric", "")
assert metric in ("", "loss"), f"TabNet metric must be '' or 'loss', got {metric!r}"

Type guard

def is_supported_tabnet_metric(metric: str) -> bool:
    return metric in ("", "loss")

Try / catch

try:
    model.fit(dataset)
except ValueError as e:
    if "unknown metric" in str(e):
        raise ValueError("TabNet early stopping tracks only the loss; use metric='' or 'loss'") from e
    raise

Prevention

When it happens

Trigger: metric='ic' (or any unsupported token) in TabNet kwargs, then fit(); the raise occurs mid-training at the first evaluation step.

Common situations: Porting metric names from other qlib examples; expecting sklearn/pytorch-tabnet metric names to carry over.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/e13e92da5e57a6eb. Report an issue: GitHub.