unmount of {} could not be verified (still mounted or mount

Error message

unmount of {} could not be verified (still mounted or mount table inconclusive); retaining backing and pin

What it means

After plain_umount, the code re-checks dest_is_known_unmounted to confirm the dest is really no longer mounted. If the unmount could not be verified (still listed in the mount table, or the table is unreadable), it refuses to proceed and keeps the backing directory and pin intact rather than deleting data beneath a possibly-live mount.

Source

Thrown at crates/codegen/xai-fast-worktree/src/nfs/remove.rs:56

    if client.ping() {
        match client.remove_worktree(worktree_path, false) {
            Ok(()) => return report_after_daemon_unmount(worktree_path),
            Err(e) => {
                bail!("daemon RemoveWorktree failed: {e}");
            }
        }
    }
    if dest_is_mountpoint(worktree_path) {
        if lookup_from_markers(worktree_path).is_none() {
            bail!(
                "{} is still a mountpoint without a grove marker; refusing umount/rm",
                worktree_path.display()
            );
        }
        plain_umount(worktree_path)?;
    }
    if !super::dest_is_known_unmounted(worktree_path) {
        bail!(
            "unmount of {} could not be verified (still mounted or mount table \
             inconclusive); retaining backing and pin",
            worktree_path.display()
        );
    }
    let meta = lookup_nfs_meta(worktree_path);
    if let Some(m) = meta.as_ref() {
        let Some(id) = m.worktree_id.as_deref() else {
            bail!(
                "unmounted dest {} has grove metadata without a worktree id; \
                 retaining pin and dest",
                worktree_path.display()
            );
        };
        if !is_safe_worktree_id(id) {
            bail!(
                "unmounted dest {} has unsafe worktree id {id:?}; retaining pin and dest",
                worktree_path.display()

View on GitHub (pinned to bc7f02eddd)

Solutions

Find and stop processes using the mount (lsof/fuser -v <path>), then retry removal
Use umount -l (lazy) or unmount submounts first, then re-verify the table
Run in the correct mount namespace (e.g. via nsenter) so the mount table is readable and accurate
Re-run try_nfs_remove once the table shows the dest unmounted

Example fix

// before
Command::new("umount").arg(&path).status()?;
remove_backing_and_pin(&path)?; // may delete under live mount
// after
Command::new("fuser").arg("-k").arg("-m").arg(&path).status()?;
Command::new("umount").arg(&path).status()?;
if dest_is_known_unmounted(&path) { remove_backing_and_pin(&path)?; }

Defensive patterns

Strategy: retry

Validate before calling

fn unmounted(path: &Path) -> bool {
    super::dest_is_known_unmounted(path)
}
if !unmounted(&path) {
    let _ = Command::new("fuser").arg("-k").arg("-m").arg(&path).status();
    let _ = Command::new("umount").arg(&path).status();
}

Try / catch

match try_nfs_remove(&path) {
    Err(e) if e.to_string().contains("could not be verified") => {
        // kill holders, lazy-umount submounts, wait, then retry once
    }
    r => r?,
}

Prevention

Ensure no process holds fds or cwd inside the mount
Unmount nested submounts before the dest
Run in the mount namespace where the mount exists so verification is accurate

When it happens

Trigger: plain_umount reported success (or was skipped) but the mount still appears in the mount table — lazy mounts, open file descriptors keeping it alive, submounts still attached — or dest_is_known_unmounted cannot confirm state.

Common situations: Processes (shells, editors, build daemons) holding cwd/fds inside the mount; umount returning EBUSY handled loosely; containers/sandboxes where /proc/mounts of the namespace isn't visible to the tool.

Related errors

AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/10a7538d138e83da. Report an issue: GitHub.