What causes "Unknown criterion: {self.criterion}" in microsoft/qlib?

Use criterion="mse" for plain regression loss on meta labels.

microsoft/qlib · error · ValueError

Unknown criterion: {self.criterion}

Error message

Unknown criterion: {self.criterion}

What it means

MetaModelDS.train_only_logs/test loop computes its loss with either nn.MSELoss (criterion='mse') or the custom ICLoss (criterion='ic_loss'). Any other criterion string reaches the else and raises ValueError, because no other loss constructor is wired in.

Source

Thrown at qlib/contrib/meta/data_selection/model.py:104

                meta_input["X"],
                meta_input["y"],
                meta_input["time_perf"],
                meta_input["time_belong"],
                meta_input["X_test"],
                ignore_weight=ignore_weight,
            )
            if self.criterion == "mse":
                criterion = nn.MSELoss()
                loss = criterion(pred, meta_input["y_test"])
            elif self.criterion == "ic_loss":
                criterion = ICLoss(self.loss_skip_thresh)
                try:
                    loss = criterion(pred, meta_input["y_test"], meta_input["test_idx"])
                except ValueError as e:
                    get_module_logger("MetaModelDS").warning(f"Exception `{e}` when calculating IC loss")
                    continue
            else:
                raise ValueError(f"Unknown criterion: {self.criterion}")

            assert not np.isnan(loss.detach().item()), "NaN loss!"

            if phase == "train":
                opt.zero_grad()
                loss.backward()
                opt.step()
            elif phase == "test":
                pass

            pred_y_all.append(
                pd.DataFrame(
                    {
                        "pred": pd.Series(pred.detach().cpu().numpy(), index=meta_input["test_idx"]),
                        "label": pd.Series(meta_input["y_test"].detach().cpu().numpy(), index=meta_input["test_idx"]),
                    }
                )
            )

View on GitHub (pinned to 79633dd950)

Solutions

Use criterion="mse" for plain regression loss on meta labels.
Use criterion="ic_loss" to optimize the (negative) information-coefficient loss over daily cross-sections.
If you need another loss, subclass MetaModelDS and add a branch constructing your nn.Module criterion.

Example fix

// before
model = MetaModelDS(..., criterion="IC_loss")  # trains then raises

// after
model = MetaModelDS(..., criterion="ic_loss")

Defensive patterns

Strategy: validation

Validate before calling

if criterion not in ("mse", "ic_loss"):
    raise ValueError(f"MetaModelDS criterion must be 'mse' or 'ic_loss', got {criterion!r}")
model = MetaModelDS(..., criterion=criterion)

Type guard

def is_meta_criterion(c) -> bool:
    return c in ("mse", "ic_loss")

Try / catch

try:
    model.fit(...)
except ValueError as e:
    if "Unknown criterion" in str(e):
        raise ValueError("criterion must be 'mse' or 'ic_loss'") from e
    raise

Prevention

Validate criterion at construction time, since MetaModelDS defers the error to training.
Note 'ic_loss' additionally skips batches whose IC calc fails (logged warning).
Subclass to add losses rather than passing unsupported names.

When it happens

Trigger: Constructing MetaModelDS(..., criterion='mae') or 'cross_entropy' etc.; the constructor accepts the string without validating it, so the error surfaces only when training starts.

Common situations: Trying standard PyTorch loss names on the meta model; typo like 'IC_loss' or 'ICLoss'; copy-pasting criterion configs from other qlib model classes.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/a66c25096f86b87a. Report an issue: GitHub.