gastownhall/beads · warning

timeout (%s) waiting for spawn marker %s; wait for the in-pr

Error message

timeout (%s) waiting for spawn marker %s; wait for the in-progress start to finish, then retry

What it means

During stopAndAcquire, after acquiring proxy.lock the shutdown path checks the spawn marker: if a start attempt is still in progress (marker active), it unlocks and polls until the marker clears. If the marker remains active past shutdownConfirmDeadline (5s), Shutdown returns this timeout telling the caller a start is mid-flight and to wait and retry. This is intentional backpressure, not a process-management failure.

Source

Thrown at internal/storage/dbproxy/proxy/shutdown.go:180

	lockPath := filepath.Join(rootDir, lockName)
	recordPath := pidfile.Path(rootDir, pidName)
	deadline := time.Now().Add(shutdownConfirmDeadline)
	var stopped *pidfile.PidFile

	for {
		lock, err := util.TryLock(lockPath)
		switch {
		case err == nil:
			if checks.CheckSpawnMarker {
				active, markerErr := inspectSpawnMarkerLocked(rootDir)
				if markerErr != nil {
					lock.Unlock()
					return nil, markerErr
				}
				if active {
					lock.Unlock()
					if time.Now().After(deadline) {
						return nil, fmt.Errorf(
							"timeout (%s) waiting for spawn marker %s; wait for the in-progress start to finish, then retry",
							shutdownConfirmDeadline,
							filepath.Join(rootDir, spawnMarkerFileName),
						)
					}
					time.Sleep(shutdownConfirmPoll)
					continue
				}
			}

			pf, readErr := pidfile.Read(rootDir, pidName)
			if readErr != nil {
				lock.Unlock()
				if isMalformedPIDFileError(readErr) {
					return nil, unverifiableProcessError(
						"shutdown",
						recordPath,
						0,

View on GitHub (pinned to 71377f2769)

Solutions

1. Wait for the in-progress start to finish, then re-run the shutdown (as the message states).
2. Check whether a bd/dolt start process is actually alive (ps aux | grep -E 'bd|dolt'); if it was killed, the marker is stale.
3. If the marker is stale (no live start), remove <rootDir>/<spawn marker file> and retry the shutdown.
4. Re-run with backoff in scripts: retry the stop a few times over ~30s before failing.
5. Avoid racing start/stop in automation — serialize them or wait on the start's exit before stopping.

Example fix

// before (script): immediate stop races start
bd dolt start &
bd dolt stop   # timeout (5s) waiting for spawn marker ...
// after
bd dolt start &
wait $! || true
bd dolt stop

// Before stopping, check whether a start is still marked in progress
marker := filepath.Join(rootDir, spawnMarkerFileName)
if _, err := os.Stat(marker); err == nil {
    // a start may be in flight; defer the stop or verify no live start process exists
}

err := proxy.Shutdown(rootDir)
for i := 0; i < 5 && err != nil && strings.Contains(err.Error(), "waiting for spawn marker"); i++ {
    time.Sleep(3 * time.Second)
    err = proxy.Shutdown(rootDir)
}
if err != nil && strings.Contains(err.Error(), "waiting for spawn marker") {
    // likely stale marker: verify no live start process, then remove the marker and retry
}

When it happens

Trigger: Calling proxy.Shutdown while a bd start (spawn marked attempt) has held the spawn marker longer than 5 seconds — e.g. a very slow backend start, a hung spawn, or a crash that left a stale spawn marker behind.

Common situations: Racing 'bd dolt start' and 'bd dolt stop' from different shells/scripts; backend start slowed by cold cache or heavy IO; a previous start was SIGKILLed leaving an orphaned spawn marker that never clears.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• errStartInterrupted
• timeout waiting for cache lock on %s
• timeout (%s) acquiring %s after inspecting pid %d at %s; sto
• acquire gc connection: %w
• ErrLockHeld

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/f8d52a9860096ff8. Report an issue: GitHub.