finish_open: could not acquire shared lock on data file at

Error message

finish_open: could not acquire shared lock on data file at {:?}

What it means

finish_open must acquire a shared advisory file lock (flock) on the cached data file to read it safely; if try_lock_shared fails it panics. This guards against concurrent eviction/modification while reading. Failure indicates lock contention with an evicting writer or an OS/filesystem that doesn't support the lock.

Solutions

Retry the read; transient contention with the eviction task usually resolves quickly
Avoid sharing the cache directory across filesystems that don't support flock (use a local disk via POLARS_TEMP_DIR)
Ensure no other process holds an exclusive lock on the cached file
Disable/limit concurrent eviction or use separate cache dirs per job

Defensive patterns

Strategy: retry

Validate before calling

// Pre-check: ensure cache lives on a local filesystem that supports flock
let base = std::env::var("POLARS_TEMP_DIR").unwrap_or_else(|_| std::env::temp_dir().display().to_string());
assert!(!base.starts_with("/mnt/nfs"), "file cache on NFS may not support flock; use a local disk");

Try / catch

// The API panics rather than returning Err; catch at process level or retry the job
for attempt in 0..3 {
    let result = std::panic::catch_unwind(|| read_cached(path));
    if result.is_ok() { break; }
    std::thread::sleep(std::time::Duration::from_millis(200 * (attempt + 1)));
}

Prevention

Keep the file cache on local disk, not NFS/SMB
Avoid many concurrent processes sharing one cache directory with aggressive eviction
Retry transient lock contention at the job level

When it happens

Trigger: Opening a cached cloud file (try_open_assume_latest / try_open_check_latest) while another process/thread holds an exclusive lock (eviction in progress), or on filesystems without flock support (some network mounts).

Common situations: NFS/SMB mounts lacking flock, concurrent Polars processes racing eviction, or lock-holder processes killed leaving inconsistent state on exotic filesystems.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of pola-rs/polars@fe841f959e (2026-09-18). Data as JSON: /api/errors/7349a04264d08832. Report an issue: GitHub.

Appendix: source

Thrown at crates/polars-io/src/file_cache/entry.rs:393

        {
            std::fs::OpenOptions::new()
                .read(true)
                .open(data_file_path)
                .unwrap()
        }
        // windows requires write access to update the last accessed time
        #[cfg(target_family = "windows")]
        {
            std::fs::OpenOptions::new()
                .read(true)
                .write(true)
                .open(data_file_path)
                .unwrap()
        }
    };
    update_last_accessed(&file);
    if FileExt::try_lock_shared(&file).is_err() {
        panic!(
            "finish_open: could not acquire shared lock on data file at {:?}",
            data_file_path
        );
    }
    file
}

/// `[prefix]/d/[uri hash][last modified]`
fn get_data_file_path(
    path_prefix: &[u8],
    uri_hash: &[u8],
    remote_version: &FileVersion,
) -> PathBuf {
    let owned;
    let path = [
        path_prefix,
        &[b'/', DATA_PREFIX, b'/'],
        uri_hash,

View on GitHub (pinned to fe841f959e)