{"record":{"id":"3c2e80932f8d0632","repo":"microsoft/qlib","slug":"empty-data-from-dataset-please-check-your-dataset-3c2e80","errorCode":null,"errorMessage":"Empty data from dataset, please check your dataset config.","messagePattern":"Empty data from dataset, please check your dataset config\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_general_nn.py","lineNumber":249,"sourceCode":"                scores.append(score.item())\n\n        return np.mean(losses), np.mean(scores)\n\n    def fit(\n        self,\n        dataset: Union[DatasetH, TSDatasetH],\n        evals_result=dict(),\n        save_path=None,\n        reweighter=None,\n    ):\n        ists = isinstance(dataset, TSDatasetH)  # is this time series dataset\n\n        dl_train = dataset.prepare(\"train\", col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\n        dl_valid = dataset.prepare(\"valid\", col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\n        self.logger.info(f\"Train samples: {len(dl_train)}\")\n        self.logger.info(f\"Valid samples: {len(dl_valid)}\")\n        if dl_train.empty or dl_valid.empty:\n            raise ValueError(\"Empty data from dataset, please check your dataset config.\")\n\n        if reweighter is None:\n            wl_train = np.ones(len(dl_train))\n            wl_valid = np.ones(len(dl_valid))\n        elif isinstance(reweighter, Reweighter):\n            wl_train = reweighter.reweight(dl_train)\n            wl_valid = reweighter.reweight(dl_valid)\n        else:\n            raise ValueError(\"Unsupported reweighter type.\")\n\n        # Preprocess for data.  To align to Dataset Interface for DataLoader\n        if ists:\n            dl_train.config(fillna_type=\"ffill+bfill\")  # process nan brought by dataloader\n            dl_valid.config(fillna_type=\"ffill+bfill\")  # process nan brought by dataloader\n        else:\n            # If it is a tabular, we convert the dataframe to numpy to be indexable by DataLoader\n            dl_train = dl_train.values\n            dl_valid = dl_valid.values","sourceCodeStart":231,"sourceCodeEnd":267,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_general_nn.py#L231-L267","documentation":"Raised at the top of DNNModelPytorch.fit when either the \"train\" or \"valid\" segment prepared from the dataset is an empty DataFrame. The model requires both segments to be non-empty before it builds DataLoaders, so an empty validation split is fatal here (unlike some other qlib models that tolerate a missing valid segment). It is a dataset/segment configuration guard, not a training-time failure.","triggerScenarios":"Calling fit(dataset) where dataset.prepare(\"train\" or \"valid\", col_set=[\"feature\",\"label\"], data_key=DK_L) returns zero rows: date ranges that do not intersect the data calendar, a \"valid\" segment key not defined in the dataset, or handlers whose learn-process dropped all rows.","commonSituations":"Segments specified outside the data's date range (e.g. backtest dates beyond the dump_bin calendar); mislabelled segment names (\"validation\" instead of \"valid\"); a filter/processor that drops every row; using a dataset where only train and test segments were defined, forgetting that this model demands a valid segment.","solutions":["Check the log lines 'Train samples: N / Valid samples: N' just above the raise; whichever is 0 is the broken segment.","Align dataset segments (train/valid/test date ranges) with the calendar of your dumped bin data.","Ensure the segment is named \"valid\" in DatasetH.setup_seg or kwargs so dfs includes it.","Verify processors/fillna are not removing all rows (inspect dataset.prepare(..., data_key=DataHandlerLP.DK_L).shape)."],"exampleFix":"# before\n\"segments\": {\n    \"train\": (\"2015-01-01\", \"2020-12-31\"),\n    \"valid\": (\"2021-01-01\", \"2021-12-31\"),  # calendar ends 2020 -> empty\n}\n\n# after\n\"segments\": {\n    \"train\": (\"2015-01-01\", \"2019-12-31\"),\n    \"valid\": (\"2020-01-01\", \"2020-12-31\"),\n}","handlingStrategy":"validation","validationCode":"for seg in (\"train\", \"valid\"):\n    df = dataset.prepare(seg, col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\n    assert not df.empty, f\"segment '{seg}' is empty; check dates/calendar/processors\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always print len() of each prepared segment before fit; the model logs 'Train/Valid samples' for you.","Keep segment date ranges inside the calendar produced by dump_bin.","Ensure both 'train' and 'valid' segments are defined — this model requires both."],"tags":["qlib","dataset","configuration","data-validation"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}