Caching for the dataset is not enabled, try `dataset.purge_c

Error message

Caching for the dataset is not enabled, try `dataset.purge_cache()` if possible. The dataset is {dataset}.

What it means

compute_lens warns when computing sample lengths for a task whose dataset has no cache enabled — every epoch will re-run expensive preprocessing. It suggests purge_cache()/enabling caching for performance, not correctness.

Source

Thrown at hanlp/components/mtl/tasks/__init__.py:166

        pass

    # noinspection PyMethodMayBeStatic
    def compute_lens(self, data: Union[List[Dict[str, Any]], str], dataset: TransformableDataset,
                     input_ids='token_input_ids'):
        """

        Args:
            data: Samples to be measured or path to dataset during training time.
            dataset: During training time, use this dataset to measure the length of each sample inside.
            input_ids: Field name corresponds to input ids.

        Returns:

            Length list of this samples

        """
        if dataset.cache is None:
            warnings.warn(f'Caching for the dataset is not enabled, '
                          f'try `dataset.purge_cache()` if possible. The dataset is {dataset}.')
        if isinstance(data, str):
            timer = CountdownTimer(len(dataset))
            for each in dataset:
                timer.log('Preprocessing and caching samples [blink][yellow]...[/yellow][/blink]')
            timer.erase()
        return [len(x[input_ids]) for x in dataset]

    def feed_batch(self,
                   h: torch.FloatTensor,
                   batch: Dict[str, torch.Tensor],
                   mask: torch.BoolTensor,
                   decoder: torch.nn.Module):
        return decoder(h, batch=batch, mask=mask)

    def input_is_flat(self, data) -> bool:
        """
        Check whether the data is flat (meaning that it's only a single sample, not even batched).

View on GitHub (pinned to ddb1299bdd)

Solutions

Enable caching on the dataset (pass cache-specific args / call dataset.purge_cache() then rebuild so a cache is written) as the message suggests
Compute and pass lens/samples explicitly if the API allows, skipping the iteration
Accept the one-time cost if the dataset is small — the warning is performance-only

Example fix

# before
dl = task.build_dataloader(data, dataset=TableDataset(data))  # no cache -> warning
# after
ds = TableDataset(data, cache='cache.pkl')
dl = task.build_dataloader(data, dataset=ds)

Defensive patterns

Strategy: validation

Validate before calling

if dataset.cache is None:
    dataset.cache = 'cache.pkl'  # enable caching before build_dataloader

Prevention

Pass a cache path when constructing datasets for MTL tasks
Call dataset.purge_cache() after data changes, then rebuild

When it happens

Trigger: Building a dataloader for an MTL task whose samples lack cached lengths (dataset.cache is None) but lengths must be computed by iterating and preprocessing every sample.

Common situations: Loading a large MTL dataset without cache_path/enable_cache; reading from a non-cacheable streaming dataset; first run before a cache file exists.

Related errors

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/f01340062413bcb8. Report an issue: GitHub.