{"record":{"id":"b7e976d44e754429","repo":"clockworklabs/SpacetimeDB","slug":"repo","errorCode":null,"errorMessage":"repo {}: {}: {}","messagePattern":"repo (.+?): (.+?): (.+?)","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"crates/commitlog/src/repo/fs.rs","lineNumber":243,"sourceCode":"            Ok(len)\n        }\n    }\n}\n\nimpl Repo for Fs {\n    type SegmentWriter = File;\n    type SegmentReader = ReadOnlySegment;\n\n    fn create_segment(&self, offset: u64, header: segment::Header) -> io::Result<Self::SegmentWriter> {\n        let path = self.segment_path(offset);\n\n        // We need to check if the segment already exists,\n        // so use file locking to prevent a TOCTOU race.\n        // Using `flock` means we don't need to worry about stale lockfiles.\n        let lock_path = path.0.with_extension(\"lock\");\n        let _lock = scopeguard::guard(\n            lockfile::advisory::LockedFile::lock(&lock_path)\n                .map_err(|e| io::Error::new(e.source.kind(), format!(\"repo {}: {}: {}\", self, e, e.source)))?,\n            |lockfile| {\n                if let Err(e) = lockfile.release(true) {\n                    // It's ok if removing the file fails, but print a warning\n                    // anyways.\n                    warn!(\"repo {}: failed to remove {}: {}\", self, lock_path.display(), e);\n                }\n            },\n        );\n\n        // Check whether the segment already exists.\n        // Overwrite it if its length is zero.\n        match fs::metadata(&path) {\n            Ok(stat) => {\n                if stat.len() > 0 {\n                    return Err(io::Error::new(\n                        io::ErrorKind::AlreadyExists,\n                        format!(\"repo {}: segment {} already exists and is non-empty\", self, offset),\n                    ));","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/clockworklabs/SpacetimeDB/blob/524b4487d949b61a07d4f39c862d1290259dfd20/crates/commitlog/src/repo/fs.rs#L225-L261","documentation":"Raised inside create_segment when acquiring the advisory lock file (<segment>.lock) fails. The message is 'repo <dir>: <lock error>: <cause>' and the ErrorKind is taken from the underlying cause, so the actionable detail is in the inner text. Lock acquisition happens whenever the log initializes or rolls to a new segment.","triggerScenarios":"The log directory is not writable (PermissionDenied while creating <segment>.lock); the lock file is held by another live process; the filesystem is full or the process exhausted its file-descriptor limit while creating the lock file.","commonSituations":"Two instances pointed at the same directory; containers/read-only mounts with dropped write permission; ENOSPC or EMFILE under heavy load right when a segment rolls.","solutions":["Read the inner cause in the message: check write permissions on the directory, free disk space, and the process fd limit","Ensure exactly one writer process per log directory","Make the directory writable, then retry the commit/open that triggered segment creation","If another process holds the lock, stop that process; flock locks die with their holder, so a stuck lock usually means a live holder - do not delete lock files"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"fn dir_ready_for_log(dir: &std::path::Path) -> io::Result<()> {\n    let meta = std::fs::metadata(dir)?;\n    if meta.is_dir() && !meta.permissions().readonly() {\n        Ok(())\n    } else {\n        Err(io::Error::new(io::ErrorKind::PermissionDenied, \"log dir missing or read-only\"))\n    }\n}","typeGuard":"fn is_segment_lock_error(e: &io::Error) -> bool {\n    e.to_string().contains(\".lock\")\n}","tryCatchPattern":"// transient lock contention or fd pressure: back off and retry\nlet mut backoff = std::time::Duration::from_millis(50);\nloop {\n    match attempt {\n        Ok(v) => break v,\n        Err(e) if is_segment_lock_error(&e) && backoff < std::time::Duration::from_secs(5) => {\n            std::thread::sleep(backoff);\n            backoff *= 2;\n        }\n        Err(e) => return Err(e),\n    }\n}","preventionTips":["Run exactly one writer process per log directory","Pre-flight check that the directory is writable and has free space before opening the log","Monitor open file-descriptor counts on services that create logs frequently"],"tags":["rust","commitlog","file-lock","segment-creation","permissions"],"backgroundTag":"file-lock-acquisition-failed","analyzedSha":"524b4487d949b61a07d4f39c862d1290259dfd20","analyzedAt":"2026-08-16T23:58:54.611Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}