quickwit-oss/tantivy · error

Failed to spawn meta file watcher thread

Error message

Failed to spawn meta file watcher thread

What it means

Fires in FileWatcher::spawn when thread::Builder::spawn fails to create the background polling thread that watches the meta file (e.g. meta.json) for checksum changes. Typical causes are OS resource exhaustion (thread creation limits, out of memory). Without this thread, watchers registered via watch() would never be notified of meta file changes, so the code treats a failed spawn as fatal instead of silently losing change notifications.

Source

Thrown at src/directory/mmap_directory/file_watcher.rs:65

                while state.load(Ordering::SeqCst) == 1 {
                    if let Ok(checksum) = FileWatcher::compute_checksum(&path) {
                        let metafile_has_changed = current_checksum_opt
                            .map(|current_checksum| current_checksum != checksum)
                            .unwrap_or(true);
                        if metafile_has_changed {
                            debug!("Meta file {path:?} was modified");
                            current_checksum_opt = Some(checksum);
                            // We actually ignore callbacks failing here.
                            // We just wait for the end of their execution.
                            let _ = callbacks.broadcast().wait();
                        }
                    }

                    thread::sleep(POLLING_INTERVAL);
                }
            })
            .expect("Failed to spawn meta file watcher thread");
    }

    pub fn watch(&self, callback: WatchCallback) -> WatchHandle {
        let handle = self.callbacks.subscribe(callback);
        self.spawn();
        handle
    }

    fn compute_checksum(path: &Path) -> Result<u32, io::Error> {
        let reader = match fs::File::open(path) {
            Ok(f) => io::BufReader::new(f),
            Err(e) => {
                warn!("Failed to open meta file {path:?}: {e:?}");
                return Err(e);
            }
        };

        let mut hasher = Hasher::new();

View on GitHub (pinned to b5d8deb80c)

Solutions

1. Check system thread limits (ulimit) and available memory
2. Reduce the number of concurrently watched directories
3. Handle the JoinError/IOError from spawn gracefully and fall back to manual polling if a hard failure is undesirable

When it happens

Trigger: Thrown at src/directory/mmap_directory/file_watcher.rs:65 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of quickwit-oss/tantivy@b5d8deb80c (2026-09-05). Data as JSON: /api/errors/b43ec23543a8b2f2. Report an issue: GitHub.