{"record":{"id":"a6bff1d701b553d6","repo":"microsoft/qlib","slug":"empty-data-from-dataset-please-check-your-dataset-a6bff1","errorCode":null,"errorMessage":"Empty data from dataset, please check your dataset config.","messagePattern":"Empty data from dataset, please check your dataset config\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/highfreq_gdbt_model.py","lineNumber":86,"sourceCode":"\n        res = pd.Series(self.model.predict(x_test.values), index=x_test.index)\n        y_test[\"pred\"] = res\n\n        up_p, down_p, up_a, down_a = self._cal_signal_metrics(y_test, threhold, 1 - threhold)\n        print(\"===============================\")\n        print(\"High frequency signal test\")\n        print(\"===============================\")\n        print(\"Test set precision: \")\n        print(\"Positive precision: {}, Negative precision: {}\".format(up_p, down_p))\n        print(\"Test Alpha Average in test set: \")\n        print(\"Positive average alpha: {}, Negative average alpha: {}\".format(up_a, down_a))\n\n    def _prepare_data(self, dataset: DatasetH):\n        df_train, df_valid = dataset.prepare(\n            [\"train\", \"valid\"], col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L\n        )\n        if df_train.empty or df_valid.empty:\n            raise ValueError(\"Empty data from dataset, please check your dataset config.\")\n\n        x_train, y_train = df_train[\"feature\"], df_train[\"label\"]\n        x_valid, y_valid = df_valid[\"feature\"], df_valid[\"label\"]\n        if y_train.values.ndim == 2 and y_train.values.shape[1] == 1:\n            l_name = df_train[\"label\"].columns[0]\n            # Convert label into alpha\n            df_train.loc[:, (\"label\", l_name)] = (\n                df_train.loc[:, (\"label\", l_name)]\n                - df_train.loc[:, (\"label\", l_name)].groupby(level=0, group_keys=False).mean()\n            )\n            df_valid.loc[:, (\"label\", l_name)] = (\n                df_valid.loc[:, (\"label\", l_name)]\n                - df_valid.loc[:, (\"label\", l_name)].groupby(level=0, group_keys=False).mean()\n            )\n\n            def mapping_fn(x):\n                return 0 if x < 0 else 1\n","sourceCodeStart":68,"sourceCodeEnd":104,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/highfreq_gdbt_model.py#L68-L104","documentation":"Thrown by HFMLGBModel._prepare_data when dataset.prepare returns an empty 'train' or 'valid' segment at learn time. This high-frequency model converts labels into cross-sectional alphas (subtracting per-timestamp group means) and then binarizes them into an up/down classification target, which requires non-empty train and valid frames.","triggerScenarios":"Calling fit with empty train/valid segments after DK_L processing — e.g. date ranges outside the calendar, an empty instrument universe, or processors dropping all rows.","commonSituations":"High-frequency (minute-bar) datasets with a narrower calendar than configured; segment dates not overlapping the loaded bins; aggressive NaN filtering on sparse intraday features.","solutions":["Check dataset.prepare(seg, col_set=[\"feature\",\"label\"], data_key=\"learn\").shape for 'train' and 'valid' before fit","Align handler start_time/end_time with the high-frequency calendar data actually loaded","Loosen or fix processors/instrument filters that eliminate all rows"],"exampleFix":"# before\nmodel.fit(dataset)  # ValueError: Empty data from dataset\n\n# after\nfor seg in [\"train\", \"valid\"]:\n    assert not dataset.prepare(seg, col_set=[\"feature\",\"label\"], data_key=\"learn\").empty, seg\nmodel.fit(dataset)","handlingStrategy":"validation","validationCode":"for seg in [\"train\", \"valid\"]:\n    df = dataset.prepare(seg, col_set=[\"feature\",\"label\"], data_key=\"learn\")\n    assert not df.empty, f\"{seg} empty; check high-frequency dataset config\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify intraday calendar coverage matches handler dates before fit","Watch for sparse minute-bar features being fully dropped by NaN processors"],"tags":["high-frequency","dataset-config","empty-data","qlib"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}