{"record":{"id":"1a9b3bce259eac14","repo":"microsoft/qlib","slug":"empty-data-from-dataset-please-check-your-dataset-1a9b3b","errorCode":null,"errorMessage":"Empty data from dataset, please check your dataset config.","messagePattern":"Empty data from dataset, please check your dataset config\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/contrib/model/pytorch_transformer_ts.py","lineNumber":147,"sourceCode":"                loss = self.loss_fn(pred, label)\r\n                losses.append(loss.item())\r\n\r\n                score = self.metric_fn(pred, label)\r\n                scores.append(score.item())\r\n\r\n        return np.mean(losses), np.mean(scores)\r\n\r\n    def fit(\r\n        self,\r\n        dataset: DatasetH,\r\n        evals_result=dict(),\r\n        save_path=None,\r\n    ):\r\n        dl_train = dataset.prepare(\"train\", col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\r\n        dl_valid = dataset.prepare(\"valid\", col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\r\n\r\n        if dl_train.empty or dl_valid.empty:\r\n            raise ValueError(\"Empty data from dataset, please check your dataset config.\")\r\n\r\n        dl_train.config(fillna_type=\"ffill+bfill\")  # process nan brought by dataloader\r\n        dl_valid.config(fillna_type=\"ffill+bfill\")  # process nan brought by dataloader\r\n\r\n        train_loader = DataLoader(\r\n            dl_train, batch_size=self.batch_size, shuffle=True, num_workers=self.n_jobs, drop_last=True\r\n        )\r\n        valid_loader = DataLoader(\r\n            dl_valid, batch_size=self.batch_size, shuffle=False, num_workers=self.n_jobs, drop_last=True\r\n        )\r\n\r\n        save_path = get_or_create_path(save_path)\r\n\r\n        stop_steps = 0\r\n        train_loss = 0\r\n        best_score = -np.inf\r\n        best_epoch = 0\r\n        evals_result[\"train\"] = []\r","sourceCodeStart":129,"sourceCodeEnd":165,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/contrib/model/pytorch_transformer_ts.py#L129-L165","documentation":"Raised by TransformerTSModel.fit when either the 'train' or 'valid' segment prepared from the DatasetH is empty (zero rows) after calling dataset.prepare with col_set=['feature','label'] and DataHandlerLP.DK_L. The model cannot train on an empty dataframe, so it fails fast rather than crashing later inside the PyTorch training loop. It almost always points to a dataset/segment configuration problem, not a model problem.","triggerScenarios":"Calling fit() on a DatasetH whose 'train' or 'valid' segment has no data: date segments that don't overlap the underlying qlib calendar/data range, wrong segment keys in handlers (e.g. missing 'train' or 'valid'), an instrument universe with no valid stock data in the segment window, or a data handler config that filters out all rows under DK_L.","commonSituations":"Copying an example config (e.g. alpha158 workflow) but changing the date range to years not present in the local qlib data dump; specifying test/start dates outside the calendar; using a custom instrument file whose instruments have no cached features; passing a dataset built with data_key that yields empty learn data.","solutions":["Inspect the segments: print(dataset.prepare('train', col_set=['feature','label'], data_key=DataHandlerLP.DK_L).shape) and the same for 'valid' to confirm which one is empty.","Verify your date segments overlap the qlib calendar loaded at init (D.calendar()) and that the local bin data covers those dates.","Check the instruments file / market expression used when creating the dataset actually resolves to instruments with data in the segment window.","Ensure the data handler was set up with the standard train/valid/test split (DatasetH with segments covering all three) rather than only 'test'."],"exampleFix":"# before\nmodel.fit(dataset)  # ValueError: Empty data from dataset\n\n# after\nfor seg in [\"train\", \"valid\"]:\n    df = dataset.prepare(seg, col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\n    print(seg, df.shape)  # find which segment is empty, fix segments/instruments\nmodel.fit(dataset)","handlingStrategy":"validation","validationCode":"from qlib.data.dataset.handler import DataHandlerLP\n\nfor seg in (\"train\", \"valid\"):\n    df = dataset.prepare(seg, col_set=[\"feature\", \"label\"], data_key=DataHandlerLP.DK_L)\n    if df.empty:\n        raise RuntimeError(f\"segment '{seg}' is empty; check segments/instruments/calendar\")\nmodel.fit(dataset)","typeGuard":null,"tryCatchPattern":"try:\n    model.fit(dataset)\nexcept ValueError as e:\n    if \"Empty data from dataset\" in str(e):\n        # log segments and calendar range, then fix dataset config\n        log.error(\"empty segment; train=%s valid=%s\", train_shape, valid_shape)\n    raise","preventionTips":["Print .shape for every prepared segment before fitting.","Assert segment date ranges overlap D.calendar() for your provider.","Validate the instrument universe resolves to non-empty data before building handlers."],"tags":["qlib","dataset","configuration","pytorch","empty-data"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}