facebook/flow · error · std::io::Error

cached canonicalize failure

Error message

cached canonicalize failure

What it means

flow_common::files::cached_canonicalize is a process-wide, sharded cache around dunce::canonicalize, built so many threads do not contend on the kernel realpath lock. It also memoizes FAILURES: when canonicalize of a path fails (almost always because the path does not exist), None is stored, and every later lookup of that exact path returns a synthetic io::Error(NotFound, "cached canonicalize failure") without touching the filesystem. Seeing it means an earlier canonicalize of this exact path failed and the negative result is still cached.

Source

Thrown at rust_port/crates/flow_common/src/files.rs:59

}

impl CanonicalizeCache {
    fn new() -> Self {
        Self {
            shards: std::array::from_fn(|_| Mutex::new(HashMap::new())),
        }
    }

    fn get_or_insert(&self, path: &Path) -> std::io::Result<PathBuf> {
        let mut hasher = std::hash::DefaultHasher::new();
        path.hash(&mut hasher);
        let shard_idx = (hasher.finish() as usize) % CANONICALIZE_CACHE_SHARDS;

        let mut shard = self.shards[shard_idx].lock().unwrap();
        if let Some(cached) = shard.get(path) {
            match cached {
                Some(p) => Ok(p.clone()),
                None => Err(std::io::Error::new(
                    std::io::ErrorKind::NotFound,
                    "cached canonicalize failure",
                )),
            }
        } else {
            // Use `dunce::canonicalize` instead of `std::fs::canonicalize`: on
            // Windows the latter returns a `\\?\` verbatim path, whose `?` is an
            // illegal filename character once the root is escaped into derived
            // paths (e.g. the server lock file), causing spurious "server already
            // running" failures. `dunce` strips the prefix when it is safe to do
            // so and is a passthrough to std on other platforms.
            let result = dunce::canonicalize(path);
            let cached_value = result.as_ref().ok().cloned();
            shard.insert(path.to_path_buf(), cached_value);
            result
        }
    }
}

View on GitHub (pinned to 5c86586199)

Solutions

1. Make sure the path exists before the first canonicalize lookup (create the file/dir, fix the root/lib configuration) — the cache only stores what the first call saw.
2. If the file was created after the failure was cached, restart the Flow process (server/CLI) so the process-lifetime cache is dropped and the path is retried.
3. Treat ErrorKind::NotFound from cached_canonicalize as 'path missing at first sighting': skip and retry later instead of propagating it as a fatal error.
4. Rule out persistent causes on that path: symlink loops, missing permissions, or a deleted parent directory (check with ls -l / readlink).

Example fix

// before: lookup happens while the file is missing; failure is cached forever
let canonical = flow_common::files::cached_canonicalize(&path)?;

// after: only canonicalize paths that exist; defer the rest to after creation
if path.try_exists()? {
    let canonical = flow_common::files::cached_canonicalize(&path)?;
} else {
    // skip / retry once the writer has created the file
}

use std::path::Path;

// Run before flow_common::files::cached_canonicalize to avoid caching a failure.
fn safe_to_canonicalize(path: &Path) -> bool {
    path.try_exists().unwrap_or(false)
}

fn is_cached_canonicalize_failure(e: &std::io::Error) -> bool {
    e.kind() == std::io::ErrorKind::NotFound
        && e.to_string().contains("cached canonicalize failure")
}

Match io::ErrorKind::NotFound separately: if the message says 'cached canonicalize failure', treat it as 'path was missing at first sighting' — skip or schedule a retry after the file is created; do not propagate as fatal corruption.

When it happens

Trigger: Calling flow_common::files::cached_canonicalize on a path that does not exist (deleted, not yet created, wrong root) at first call, then calling it again for the same path in the same process. Module resolution or file-watching code asking about a file before a build step creates it is the classic case: even after the file appears, the cached None keeps returning NotFound.

Common situations: Lookups racing file creation (watch mode during a build); files deleted and recreated at the same path; a configured root or lib dir pointing at a nonexistent directory; on Windows, paths dunce cannot canonicalize. The cache lives for the whole process, so a long-lived server keeps the negative entry until restart.

Related errors

• failed to get current directory
• failed to read input file
• failed to write flowlib file
• lock file is already held
• invalid line

AI-assisted analysis of facebook/flow@5c86586199 (2026-08-20). Data as JSON: /api/errors/0d7994903955b1aa. Report an issue: GitHub.