gastownhall/beads · error

confirm verified pid %d stopped: %w

Error message

confirm verified pid %d stopped: %w

What it means

After killing the verified recorded process, stopAndAcquire polls (waitForRecordedProcessExit) for the process to actually exit within the wait budget (at least 2s, bounded by the 5s deadline). If the process is still alive when the budget expires, this error is thrown so Shutdown doesn't hand back a lock while the old owner may still hold resources.

Source

Thrown at internal/storage/dbproxy/proxy/shutdown.go:273

				if _, quarantineErr := quarantineRecord(rootDir, pidName, time.Now()); quarantineErr != nil {
					lock.Unlock()
					return nil, fmt.Errorf("quarantine dead process record %s: %w", recordPath, quarantineErr)
				}
				return lock, nil
			}
			if killErr := handle.Kill(); killErr != nil {
				_ = handle.Close()
				lock.Unlock()
				return nil, fmt.Errorf("kill verified pid %d from %s: %w", pf.Pid, recordPath, killErr)
			}
			if closeErr := handle.Close(); closeErr != nil {
				lock.Unlock()
				return nil, fmt.Errorf("close verified process handle for pid %d: %w", pf.Pid, closeErr)
			}
			waitBudget := max(time.Until(deadline), shutdownPostKillMinimum)
			if waitErr := waitForRecordedProcessExit(pf, waitBudget); waitErr != nil {
				lock.Unlock()
				return nil, fmt.Errorf("confirm verified pid %d stopped: %w", pf.Pid, waitErr)
			}
			if removeErr := pidfile.Remove(rootDir, pidName); removeErr != nil {
				lock.Unlock()
				return nil, fmt.Errorf("remove stopped process record %s: %w", recordPath, removeErr)
			}
			return lock, nil

		case !lockfile.IsLocked(err):
			return nil, fmt.Errorf("probe %s: %w", lockPath, err)
		}

		pf, readErr := pidfile.Read(rootDir, pidName)
		if readErr != nil {
			if isMalformedPIDFileError(readErr) {
				return nil, unverifiableProcessError(
					"shutdown",
					recordPath,
					0,

View on GitHub (pinned to 71377f2769)

Solutions

  1. Check the process state with ps -o stat= -p <pid>; 'D' means uninterruptible I/O — fix the underlying storage/mount issue.
  2. Wait a moment and re-run bd shutdown; the next attempt will see it dead and quarantine/record-remove it.
  3. If it is a zombie, reap it from the parent process or exit the parent.
  4. As a last resort, remove the stale pidfile and lock manually and retry.

Example fix

// before
$ bd dolt stop
// error: confirm verified pid 4321 stopped: process did not exit within 2s
// after
$ ps -o stat= -p 4321   # inspect state; if 'Z', reap parent; if 'D', fix I/O
$ bd dolt stop          # retry
Defensive patterns

Strategy: retry

Validate before calling

pid=$(awk 'NR==1{print $1}' .beads/dolt.pid 2>/dev/null); [ -n "$pid" ] && [ "$(ps -o stat= -p $pid 2>/dev/null | tr -d ' ')" = "D" ] && echo 'process in uninterruptible I/O; fix storage first'

Try / catch

if err := proxy.Shutdown(rootDir); err != nil && strings.Contains(err.Error(), "confirm verified pid") {
    // check ps state, then retry after a short delay
    time.Sleep(2 * time.Second)
    err = proxy.Shutdown(rootDir)
}

Prevention

When it happens

Trigger: A verified proxy/backend process was SIGKILLed but remained observable (e.g. uninterruptible D state on Linux, slow kernel cleanup, zombie reaped late) past max(time.Until(deadline), 2s).

Common situations: Process stuck in uninterruptible I/O (D state) on a hung filesystem/NFS mount; extremely loaded machine delaying process teardown; zombie process not yet reaped by its parent.

Related errors

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/25ac7652c176d442. Report an issue: GitHub.