hankcs/HanLP · error · FileNotFoundError

{self.filepath}

Error message

{self.filepath}

What it means

This progress-tracking file iterator checks that its filepath still exists each time iteration starts (it may have been deleted since construction, e.g. cached files cleaned mid-run) and raises FileNotFoundError with the path otherwise.

Source

Thrown at hanlp/utils/io_util.py:647

    return cache_name, cache_valid


def merge_files(files: List[str], dst: str):
    with open(dst, 'wb') as write:
        for f in files:
            with open(f, 'rb') as read:
                shutil.copyfileobj(read, write)


class TimingFileIterator(CountdownTimer):

    def __init__(self, filepath) -> None:
        super().__init__(os.path.getsize(filepath))
        self.filepath = filepath

    def __iter__(self):
        if not os.path.isfile(self.filepath):
            raise FileNotFoundError(self.filepath)
        fp = open(self.filepath, encoding='utf-8', errors='ignore')
        line = fp.readline()
        while line:
            yield line
            self.current = fp.tell()
            line = fp.readline()
        fp.close()

    def log(self, info=None, ratio_percentage=True, ratio=True, step=0, interval=0.5, erase=True,
            logger: Union[logging.Logger, bool] = None, newline=False, ratio_width=None):
        assert step == 0
        super().log(info, ratio_percentage, ratio, step, interval, erase, logger, newline, ratio_width)

    @property
    def ratio(self) -> str:
        return f'{human_bytes(self.current)}/{human_bytes(self.total)}'

    @property

View on GitHub (pinned to ddb1299bdd)

Solutions

1. Re-generate or re-download the file before iterating.
2. If you control cleanup, delay deletion until all consumers finish (reference counting or a manifest).
3. Guard with os.path.isfile(path) before starting iteration and rebuild if missing.

Example fix

# before
it = SizeCorruptedTolerantIterable(path)
results = list(it)  # file deleted in between
# after
if not os.path.isfile(path):
    rebuild(path)
results = list(SizeCorruptedTolerantIterable(path))

import os
if not os.path.isfile(path):
    regenerate(path)  # your rebuild/re-download hook

try:
    for line in iterable: ...
except FileNotFoundError:
    regenerate(path)
    for line in SizeCorruptedTolerantIterable(path): ...

When it happens

Trigger: Constructing the iterable, then deleting/moving the file (cache eviction, tmp cleanup) before iterating; or constructing with a path that never existed (size lookup happens at __init__ via os.path.getsize, which would already fail, so this guard mostly catches deletion between init and iter).

Common situations: Long pipelines where a cleanup task removes intermediate files; multi-process readers racing a deleter; rerunning after a partial cache purge.

Related errors

• Chinese Treebank 9.0 is a copyright dataset owned by LDC whi
• The identifier {save_dir} resolves to a nonexistent meta fil
• {path}

AI-assisted analysis of hankcs/HanLP@ddb1299bdd (2026-08-27). Data as JSON: /api/errors/e30a1db6cb4ae433. Report an issue: GitHub.