xai-org/x-algorithm · error · FileNotFoundError

Lock file parent directory does not exist: {lock_file.parent

Error message

Lock file parent directory does not exist: {lock_file.parent}

What it means

RetryFileLock retries transient OSError (ENOLCK/ESTALE/ENOENT) while acquiring a file lock, but if the error is ENOENT because the lock file's parent directory no longer exists, retrying is pointless and FileNotFoundError is raised immediately naming the missing directory.

Source

Thrown at phoenix/xrex/utils/aot.py:422

        assert source.dtype == np.uint8
        buf = source
    return _replicate(buf)


@contextmanager
def RetryFileLock(lock_file: Path, *, poll_interval: float):
    assert poll_interval > 0, f"require {poll_interval=} > 0"
    lock = filelock.FileLock(lock_file, timeout=-1, poll_interval=poll_interval)

    while True:
        try:
            lock.acquire()
            break
        except OSError as e:
            if e.errno not in (errno.ENOLCK, errno.ESTALE, errno.ENOENT):
                raise
            if e.errno == errno.ENOENT and not lock_file.parent.exists():
                raise FileNotFoundError(
                    f"Lock file parent directory does not exist: {lock_file.parent}"
                ) from e
            logger.warning(
                "Transient error acquiring lock %s: %s. Retrying...",
                lock_file,
                e,
            )
            time.sleep(poll_interval)
        except NotImplementedError as e:
            if "use SoftFileLock instead" in str(e):
                logger.warning(
                    "FileSystem does not appear to support flock. Falling back to SoftFileLock for %s",
                    lock_file,
                )
                lock = filelock.SoftFileLock(lock_file, timeout=-1, poll_interval=poll_interval)
            else:
                raise

View on GitHub (pinned to 24c60942c5)

Solutions

1. Recreate the cache directory before acquiring: aot_cache_dir.mkdir(parents=True, exist_ok=True)
2. Disable or scope cleanup jobs that purge the cache directory during runs
3. Use a stable, dedicated cache directory not under /tmp or auto-purged scratch

Example fix

# before
lock = RetryFileLock(aot_cache_dir / 'lock')
# after
aot_cache_dir.mkdir(parents=True, exist_ok=True)
lock = RetryFileLock(aot_cache_dir / 'lock')

from pathlib import Path
Path(aot_cache_dir).mkdir(parents=True, exist_ok=True)  # before acquiring lock

for attempt in range(3):
    try:
        with RetryFileLock(lock_file):
            ... 
        break
    except FileNotFoundError:
        lock_file.parent.mkdir(parents=True, exist_ok=True)  # recreate and retry

When it happens

Trigger: Acquiring the AOT cache lock when the cache directory (or an ancestor like a job scratch dir) was deleted concurrently — e.g. cleanup scripts racing the job, or NFS stale mounts — between lock creation and acquisition.

Common situations: Shared caches on NFS/Lustre with aggressive purgers; tmpwatch deleting /tmp hierarchies mid-run; concurrent jobs where one removes the cache dir.

Related errors

• Cannot remap XLA compile options device assignment: assignme
• Compiler annotation file {file_path} not found.
• AOT cache directory must be provided.

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/e055974f73be3dda. Report an issue: GitHub.