{"record":{"id":"0d7994903955b1aa","repo":"facebook/flow","slug":"cached-canonicalize-failure","errorCode":null,"errorMessage":"cached canonicalize failure","messagePattern":"cached canonicalize failure","errorType":"exception","errorClass":"std::io::Error","httpStatus":null,"severity":"error","filePath":"rust_port/crates/flow_common/src/files.rs","lineNumber":59,"sourceCode":"}\n\nimpl CanonicalizeCache {\n    fn new() -> Self {\n        Self {\n            shards: std::array::from_fn(|_| Mutex::new(HashMap::new())),\n        }\n    }\n\n    fn get_or_insert(&self, path: &Path) -> std::io::Result<PathBuf> {\n        let mut hasher = std::hash::DefaultHasher::new();\n        path.hash(&mut hasher);\n        let shard_idx = (hasher.finish() as usize) % CANONICALIZE_CACHE_SHARDS;\n\n        let mut shard = self.shards[shard_idx].lock().unwrap();\n        if let Some(cached) = shard.get(path) {\n            match cached {\n                Some(p) => Ok(p.clone()),\n                None => Err(std::io::Error::new(\n                    std::io::ErrorKind::NotFound,\n                    \"cached canonicalize failure\",\n                )),\n            }\n        } else {\n            // Use `dunce::canonicalize` instead of `std::fs::canonicalize`: on\n            // Windows the latter returns a `\\\\?\\` verbatim path, whose `?` is an\n            // illegal filename character once the root is escaped into derived\n            // paths (e.g. the server lock file), causing spurious \"server already\n            // running\" failures. `dunce` strips the prefix when it is safe to do\n            // so and is a passthrough to std on other platforms.\n            let result = dunce::canonicalize(path);\n            let cached_value = result.as_ref().ok().cloned();\n            shard.insert(path.to_path_buf(), cached_value);\n            result\n        }\n    }\n}","sourceCodeStart":41,"sourceCodeEnd":77,"githubUrl":"https://github.com/facebook/flow/blob/5c865861998a8ccb7dbc82b0c1f511e9ef60c3d9/rust_port/crates/flow_common/src/files.rs#L41-L77","documentation":"flow_common::files::cached_canonicalize is a process-wide, sharded cache around dunce::canonicalize, built so many threads do not contend on the kernel realpath lock. It also memoizes FAILURES: when canonicalize of a path fails (almost always because the path does not exist), None is stored, and every later lookup of that exact path returns a synthetic io::Error(NotFound, \"cached canonicalize failure\") without touching the filesystem. Seeing it means an earlier canonicalize of this exact path failed and the negative result is still cached.","triggerScenarios":"Calling flow_common::files::cached_canonicalize on a path that does not exist (deleted, not yet created, wrong root) at first call, then calling it again for the same path in the same process. Module resolution or file-watching code asking about a file before a build step creates it is the classic case: even after the file appears, the cached None keeps returning NotFound.","commonSituations":"Lookups racing file creation (watch mode during a build); files deleted and recreated at the same path; a configured root or lib dir pointing at a nonexistent directory; on Windows, paths dunce cannot canonicalize. The cache lives for the whole process, so a long-lived server keeps the negative entry until restart.","solutions":["Make sure the path exists before the first canonicalize lookup (create the file/dir, fix the root/lib configuration) — the cache only stores what the first call saw.","If the file was created after the failure was cached, restart the Flow process (server/CLI) so the process-lifetime cache is dropped and the path is retried.","Treat ErrorKind::NotFound from cached_canonicalize as 'path missing at first sighting': skip and retry later instead of propagating it as a fatal error.","Rule out persistent causes on that path: symlink loops, missing permissions, or a deleted parent directory (check with ls -l / readlink)."],"exampleFix":"// before: lookup happens while the file is missing; failure is cached forever\nlet canonical = flow_common::files::cached_canonicalize(&path)?;\n\n// after: only canonicalize paths that exist; defer the rest to after creation\nif path.try_exists()? {\n    let canonical = flow_common::files::cached_canonicalize(&path)?;\n} else {\n    // skip / retry once the writer has created the file\n}","handlingStrategy":"validation","validationCode":"use std::path::Path;\n\n// Run before flow_common::files::cached_canonicalize to avoid caching a failure.\nfn safe_to_canonicalize(path: &Path) -> bool {\n    path.try_exists().unwrap_or(false)\n}","typeGuard":"fn is_cached_canonicalize_failure(e: &std::io::Error) -> bool {\n    e.kind() == std::io::ErrorKind::NotFound\n        && e.to_string().contains(\"cached canonicalize failure\")\n}","tryCatchPattern":"Match io::ErrorKind::NotFound separately: if the message says 'cached canonicalize failure', treat it as 'path was missing at first sighting' — skip or schedule a retry after the file is created; do not propagate as fatal corruption.","preventionTips":["Create files/dirs before resolving them; never canonicalize paths you have not observed to exist.","Remember the cache is process-wide and permanent: if a path can appear later, re-verify existence yourself before trusting a cached NotFound.","Validate configured roots and lib dirs for existence at startup, before any lookup caches a failure."],"tags":["filesystem","canonicalize","cache","not-found","rust"],"backgroundTag":"path-canonicalization-failed","analyzedSha":"5c865861998a8ccb7dbc82b0c1f511e9ef60c3d9","analyzedAt":"2026-08-20T10:41:37.992Z","contentChangedAt":"2026-08-20T10:41:37.992Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}