microsoft/qlib · error · ValueError

Empty data from dataset, please check your dataset config.

Error message

Empty data from dataset, please check your dataset config.

What it means

Thrown by DEnsembleModel.fit when dataset.prepare returns an empty train or valid segment. Double Ensemble trains a sequence of LightGBM sub-models each validated on the 'valid' segment with early stopping, so both splits must contain rows.

Source

Thrown at qlib/contrib/model/double_ensemble.py:70

        if not len(sub_weights) == num_models:
            raise ValueError("The length of sub_weights should be equal to num_models.")
        self.sub_weights = sub_weights
        self.epochs = epochs
        self.logger = get_module_logger("DEnsembleModel")
        self.logger.info("Double Ensemble Model...")
        self.ensemble = []  # the current ensemble model, a list contains all the sub-models
        self.sub_features = []  # the features for each sub model in the form of pandas.Index
        self.params = {"objective": loss}
        self.params.update(kwargs)
        self.loss = loss
        self.early_stopping_rounds = early_stopping_rounds

    def fit(self, dataset: DatasetH):
        df_train, df_valid = dataset.prepare(
            ["train", "valid"], col_set=["feature", "label"], data_key=DataHandlerLP.DK_L
        )
        if df_train.empty or df_valid.empty:
            raise ValueError("Empty data from dataset, please check your dataset config.")
        x_train, y_train = df_train["feature"], df_train["label"]
        # initialize the sample weights
        N, F = x_train.shape
        weights = pd.Series(np.ones(N, dtype=float))
        # initialize the features
        features = x_train.columns
        pred_sub = pd.DataFrame(np.zeros((N, self.num_models), dtype=float), index=x_train.index)
        # train sub-models
        for k in range(self.num_models):
            self.sub_features.append(features)
            self.logger.info("Training sub-model: ({}/{})".format(k + 1, self.num_models))
            model_k = self.train_submodel(df_train, df_valid, weights, features)
            self.ensemble.append(model_k)
            # no further sample re-weight and feature selection needed for the last sub-model
            if k + 1 == self.num_models:
                break

            self.logger.info("Retrieving loss curve and loss values...")

View on GitHub (pinned to 79633dd950)

Solutions

1. Check dataset.prepare(seg, col_set=["feature","label"], data_key="learn").shape for seg in ['train','valid'] before fit
2. Fix handler start/end times to overlap the loaded data
3. Review learn-processors for filters/dropna that empty the frame

Example fix

# before
model.fit(dataset)  # ValueError: Empty data from dataset

# after
assert all(len(dataset.prepare(s, col_set=["feature","label"], data_key="learn"]) > 0 for s in ["train", "valid"])
model.fit(dataset)

for seg in ["train", "valid"]:
    df = dataset.prepare(seg, col_set=["feature","label"], data_key="learn")
    assert not df.empty, f"segment '{seg}' is empty; check dataset config"

try:
    model.fit(dataset)
except ValueError as e:
    if "Empty data" in str(e):
        # log segment shapes and abort rather than retry
        ...
    raise

When it happens

Trigger: Calling fit with a DatasetH whose 'train' or 'valid' segment is empty after learn-time processing (DK_L), e.g. date ranges outside the calendar, an over-aggressive dropna processor, or a missing 'valid' segment.

Common situations: Same family of config mistakes as other GBM models: bad start_time/end_time, handler segments not overlapping data, processors eliminating all rows, or forgetting the valid segment is mandatory here.

Related errors

• Empty data from dataset, please check your dataset config.
• Empty data from dataset, please check your dataset config.
• Empty data from dataset, please check your dataset config.
• The length of sample_ratios should be equal to bins_fs.
• The length of sub_weights should be equal to num_models.

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/7794784516e6ba47. Report an issue: GitHub.