microsoft/qlib · error · OSError

Failed to create directory {mount_path}, please create {moun

Error message

Failed to create directory {mount_path}, please create {mount_path} manually!

What it means

CacheUtils.acquire (qlib/data/cache.py:245) wraps redis-based writer locks. redis_lock.AlreadyAcquired means the redis key 'lock:<name>-wlock' is currently held (or was left over from a crashed process), so the new acquirer gets a QlibCacheException with step-by-step instructions. It is qlib's guard against concurrent cache writers, surfaced as a RuntimeError subclass (QlibCacheException).

Source

Thrown at qlib/__init__.py:164

                        if isinstance(_c, str):
                            _temp_mount = _c.split(" ")[2]
                        else:
                            _temp_mount = _c.decode("utf-8").split(" ")[2]
                        _temp_mount = _temp_mount[:-1] if _temp_mount.endswith("/") else _temp_mount
                        if _temp_mount == _mount_path:
                            _is_mount = True
                            break
                if _is_mount:
                    break
                _remote_uri = "/".join(_remote_uri.split("/")[:-1])
                _mount_path = "/".join(_mount_path.split("/")[:-1])
                _check_level_num -= 1

            if not _is_mount:
                try:
                    Path(mount_path).mkdir(parents=True, exist_ok=True)
                except Exception as e:
                    raise OSError(
                        f"Failed to create directory {mount_path}, please create {mount_path} manually!"
                    ) from e

                # check nfs-common
                command_res = os.popen("dpkg -l | grep nfs-common")
                command_res = command_res.readlines()
                if not command_res:
                    raise OSError("nfs-common is not found, please install it by execute: sudo apt install nfs-common")
                # manually mount
                try:
                    subprocess.run(mount_command, check=True, capture_output=True, text=True)
                    LOG.info("Mount finished.")
                except subprocess.CalledProcessError as e:
                    if e.returncode == 256:
                        raise OSError("Mount failed: requires sudo or permission denied") from e
                    elif e.returncode == 32512:
                        raise OSError(f"mount {provider_uri} on {mount_path} error! Command error") from e
                    else:

View on GitHub (pinned to 79633dd950)

Solutions

  1. Follow the message: connect with redis-cli, SELECT the db shown (C.redis_task_db), DEL the exact key 'lock:<name>-wlock' listed in the error, then rerun
  2. If multiple stale keys exist, the message suggests KEYS * to enumerate and FLUSHALL (only on a redis dedicated to qlib locks!)
  3. Ensure no other qlib process is legitimately still writing the same cache before deleting the lock
  4. Consider configuring lock expiration/TTL or using a dedicated redis DB per environment to avoid collisions

Example fix

# as the error message instructs:
redis-cli
> select 1            # the db from C.redis_task_db
> del "lock:dataset-<hash>-wlock"
> quit
# then rerun your qlib command
Defensive patterns

Strategy: retry

Validate before calling

# check for a stale lock before starting a big cache job
import redis  # same config as C.redis_host/port/task_db
r = redis.Redis(host=C.redis_host, port=C.redis_port, db=C.redis_task_db)
stale = [k for k in r.keys("lock:*-wlock")]
if stale:
    print(f"stale qlib locks present: {stale}")  # decide cleanup before running

Try / catch

from qlib.data.cache import QlibCacheException

for attempt in range(3):
    try:
        run_cache_job()
        break
    except QlibCacheException as e:
        if "redis lock" not in str(e) or attempt == 2:
            raise
        clear_stale_lock(e)  # parse key from message, DEL it, backoff, retry

Prevention

When it happens

Trigger: Two processes simultaneously generating the same expression/dataset cache with redis configured as the lock backend (C.redis_host set); or a previous run was killed while holding the redis lock and the key never expired, so every subsequent run fails to acquire it.

Common situations: Multi-process data preparation (dump_bin, cache warm-up) with the same redis DB; leftover locks after OOM-kill or Ctrl-C; redis configured with no TTL on lock keys; shared redis across users/environments so key names collide.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/e07f12febb46f333. Report an issue: GitHub.