{"record":{"id":"f01340062413bcb8","repo":"hankcs/HanLP","slug":"caching-for-the-dataset-is-not-enabled-try-datas","errorCode":null,"errorMessage":"Caching for the dataset is not enabled, try `dataset.purge_cache()` if possible. The dataset is {dataset}.","messagePattern":"Caching for the dataset is not enabled, try `dataset\\.purge_cache\\(\\)` if possible\\. The dataset is (.+?)\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"hanlp/components/mtl/tasks/__init__.py","lineNumber":166,"sourceCode":"        pass\n\n    # noinspection PyMethodMayBeStatic\n    def compute_lens(self, data: Union[List[Dict[str, Any]], str], dataset: TransformableDataset,\n                     input_ids='token_input_ids'):\n        \"\"\"\n\n        Args:\n            data: Samples to be measured or path to dataset during training time.\n            dataset: During training time, use this dataset to measure the length of each sample inside.\n            input_ids: Field name corresponds to input ids.\n\n        Returns:\n\n            Length list of this samples\n\n        \"\"\"\n        if dataset.cache is None:\n            warnings.warn(f'Caching for the dataset is not enabled, '\n                          f'try `dataset.purge_cache()` if possible. The dataset is {dataset}.')\n        if isinstance(data, str):\n            timer = CountdownTimer(len(dataset))\n            for each in dataset:\n                timer.log('Preprocessing and caching samples [blink][yellow]...[/yellow][/blink]')\n            timer.erase()\n        return [len(x[input_ids]) for x in dataset]\n\n    def feed_batch(self,\n                   h: torch.FloatTensor,\n                   batch: Dict[str, torch.Tensor],\n                   mask: torch.BoolTensor,\n                   decoder: torch.nn.Module):\n        return decoder(h, batch=batch, mask=mask)\n\n    def input_is_flat(self, data) -> bool:\n        \"\"\"\n        Check whether the data is flat (meaning that it's only a single sample, not even batched).","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/hankcs/HanLP/blob/ddb1299bddff079e447af52ec12549c50636bfa8/hanlp/components/mtl/tasks/__init__.py#L148-L184","documentation":"compute_lens warns when computing sample lengths for a task whose dataset has no cache enabled — every epoch will re-run expensive preprocessing. It suggests purge_cache()/enabling caching for performance, not correctness.","triggerScenarios":"Building a dataloader for an MTL task whose samples lack cached lengths (dataset.cache is None) but lengths must be computed by iterating and preprocessing every sample.","commonSituations":"Loading a large MTL dataset without cache_path/enable_cache; reading from a non-cacheable streaming dataset; first run before a cache file exists.","solutions":["Enable caching on the dataset (pass cache-specific args / call dataset.purge_cache() then rebuild so a cache is written) as the message suggests","Compute and pass lens/samples explicitly if the API allows, skipping the iteration","Accept the one-time cost if the dataset is small — the warning is performance-only"],"exampleFix":"# before\ndl = task.build_dataloader(data, dataset=TableDataset(data))  # no cache -> warning\n# after\nds = TableDataset(data, cache='cache.pkl')\ndl = task.build_dataloader(data, dataset=ds)","handlingStrategy":"validation","validationCode":"if dataset.cache is None:\n    dataset.cache = 'cache.pkl'  # enable caching before build_dataloader","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pass a cache path when constructing datasets for MTL tasks","Call dataset.purge_cache() after data changes, then rebuild"],"tags":["dataset","performance","caching","mtl"],"backgroundTag":"cache-not-enabled-warning","analyzedSha":"ddb1299bddff079e447af52ec12549c50636bfa8","analyzedAt":"2026-08-27T03:36:54.287Z","schemaVersion":2},"datasetVersion":"2026-08-27T08:17:20.692Z"}