{"record":{"id":"66dac720eb1e5566","repo":"microsoft/qlib","slug":"empty-data-from-dataset-please-check-your-dataset-66dac7","errorCode":null,"errorMessage":"Empty data from dataset, please check your dataset config.","messagePattern":"Empty data from dataset, please check your dataset config\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_lstm.py","lineNumber":216,"sourceCode":"\n            score = self.metric_fn(pred, label)\n            scores.append(score.item())\n\n        return np.mean(losses), np.mean(scores)\n\n    def fit(\n        self,\n        dataset: DatasetH,\n        evals_result=dict(),\n        save_path=None,\n    ):\n        df_train, df_valid, df_test = dataset.prepare(\n            [\"train\", \"valid\", \"test\"],\n            col_set=[\"feature\", \"label\"],\n            data_key=DataHandlerLP.DK_L,\n        )\n        if df_train.empty or df_valid.empty:\n            raise ValueError(\"Empty data from dataset, please check your dataset config.\")\n\n        x_train, y_train = df_train[\"feature\"], df_train[\"label\"]\n        x_valid, y_valid = df_valid[\"feature\"], df_valid[\"label\"]\n\n        save_path = get_or_create_path(save_path)\n        stop_steps = 0\n        train_loss = 0\n        best_score = -np.inf\n        best_epoch = 0\n        evals_result[\"train\"] = []\n        evals_result[\"valid\"] = []\n\n        # train\n        self.logger.info(\"training...\")\n        self.fitted = True\n\n        for step in range(self.n_epochs):\n            self.logger.info(\"Epoch%d:\", step)","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_lstm.py#L198-L234","documentation":"LSTMModel.fit() prepares train/valid/test in one call, then requires df_train and df_valid to be non-empty; otherwise it raises ValueError('Empty data from dataset, please check your dataset config.') before any training. Empty usually means the segment windows do not intersect the handler's data.","triggerScenarios":"model.fit(dataset) where dataset.prepare(['train','valid','test'], col_set=['feature','label'], data_key=DK_L) yields empty 'train' or 'valid' frames — misaligned segment dates, unloaded instruments, or over-aggressive processors.","commonSituations":"Segment dates outside the handler's start_time/end_time; calendar mismatch (e.g. data ends before valid segment begins); instruments list empty after filtering; NaN-dropping processors removing all rows.","solutions":["Reproduce the emptiness: df = dataset.prepare('train', col_set=['feature','label'], data_key='learn'); print(df.shape) — do the same for 'valid'.","Adjust DatasetH segments (or the handler's start_time/end_time) so both windows contain trading dates present in the data.","Confirm the handler's underlying dataframe is non-empty (handler.fetch(col_set='feature').shape) and the instruments survive filtering.","If a processor (e.g. dropna) empties the data, relax it or widen the segments."],"exampleFix":"# before\ndataset = DatasetH(handler, segments={'train': ('2018-01-01','2018-12-31'), 'valid': ('2019-01-01','2019-12-31')})\n# handler data only covers 2020 -> df_train.empty\nmodel.fit(dataset)  # ValueError: Empty data\n\n# after\ndataset = DatasetH(handler, segments={'train': ('2020-01-01','2020-06-30'), 'valid': ('2020-07-01','2020-09-30')})\nmodel.fit(dataset)","handlingStrategy":"validation","validationCode":"df_train, df_valid, _ = dataset.prepare(\n    [\"train\", \"valid\", \"test\"], col_set=[\"feature\", \"label\"], data_key=\"learn\"\n)\nassert not df_train.empty, \"train segment is empty — check segments vs handler date range\"\nassert not df_valid.empty, \"valid segment is empty — check segments vs handler date range\"","typeGuard":null,"tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"Empty data\" in str(e):\n        # check dataset.prepare per segment; fix date windows / instruments\n        raise\n    raise","preventionTips":["Print each segment's prepared shape before fit.","Keep segment windows inside the handler's start_time/end_time coverage.","Beware calendars: windows with no trading days produce empty frames.","Check instruments survive filtering and processors don't drop all rows."],"tags":["pytorch","qlib","dataset","data-config","lstm"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}