gastownhall/beads · error

timeout waiting for proxy on %s

Error message

timeout waiting for proxy on %s

What it means

The caller gave up: for the full openDeadline window, GetCreateDatabaseProxyServerEndpoint polled for a usable proxy endpoint and none materialized, and no concrete spawn error was recorded (lastSpawnErr was nil). This is the generic 'proxy never appeared' timeout — usually the spawn marker indicates activity that never completes, or repeated lock races kept deferring the spawn until the deadline.

Source

Thrown at internal/storage/dbproxy/proxy/endpoint.go:288

		case !lockfile.IsLocked(err):
			return Endpoint{}, fmt.Errorf("probe proxy lock: %w", err)
		case discovery.status == adoptionLegacy:
			recordPath := pidfile.Path(rootDir, PIDFileName)
			return Endpoint{}, fmt.Errorf(
				"legacy proxy record %s is protected by held lock %s; stop the pre-upgrade proxy with the old bd binary or wait for its idle exit, then quarantine the record manually by renaming %s to %s.stale-<unix-timestamp> before retrying",
				recordPath,
				filepath.Join(rootDir, LockFileName),
				recordPath,
				recordPath,
			)
		}

		select {
		case <-timeout.C:
			if lastSpawnErr != nil {
				return Endpoint{}, lastSpawnErr
			}
			return Endpoint{}, fmt.Errorf("timeout waiting for proxy on %s", rootDir)
		case <-poll.C:
		}
	}
}

func adoptedEndpoint(rootDir, want string, discovery adoptionResult) (Endpoint, error) {
	if want != "" && discovery.pidfile.UpstreamID != "" && discovery.pidfile.UpstreamID != want {
		return Endpoint{}, &ErrUpstreamMismatch{
			RootDir: rootDir,
			Want:    want,
			Have:    discovery.pidfile.UpstreamID,
		}
	}
	return discovery.endpoint, nil
}

func spawnAndHandoff(
	rootDir string,

View on GitHub (pinned to 71377f2769)

Solutions

  1. Check whether a bd proxy process is actually running (pgrep -af bd); if a spawn is genuinely in progress, just wait or increase the deadline
  2. If no process is running, remove the stale spawn marker and proxy record in <rootDir> and retry
  3. Look for competing openers (CI jobs, other terminals) repeatedly locking proxy.lock and serialize them
  4. Capture a verbose log of the loop; if lastSpawnErr was swallowed, address the underlying spawn failure instead

Example fix

// before
error: timeout waiting for proxy on /repo/.beads
// after
$ pgrep -af bd || echo none
$ rm -f /repo/.beads/proxy.spawn-marker /repo/.beads/proxy.pid  # only if no bd process is alive
$ bd ready
Defensive patterns

Strategy: retry

Validate before calling

// Confirm no zombie spawn artifacts before waiting the full deadline
marker := filepath.Join(rootDir, "proxy.spawn-marker")
if fi, err := os.Stat(marker); err == nil && time.Since(fi.ModTime()) > 5*time.Minute {
    os.Remove(marker) // stale marker from a crashed opener
}

Try / catch

ep, err := proxy.GetCreateDatabaseProxyServerEndpoint(root, opts)
if err != nil && strings.Contains(err.Error(), "timeout waiting for proxy") {
    // one clean retry after clearing stale artifacts
    os.Remove(filepath.Join(rootDir, "proxy.spawn-marker"))
    ep, err = proxy.GetCreateDatabaseProxyServerEndpoint(root, opts)
}

Prevention

When it happens

Trigger: The discovery loop cycles (record absent/incomplete, spawn marker seen as active, TryLock fails as held) continuously until timeout.C fires with lastSpawnErr == nil. E.g. a previous opener crashed after creating a spawn marker without clearing it, or processes fight over proxy.lock forever.

Common situations: Stale spawn marker left by a killed/crashed opener process (SIGKILL mid-spawn); another very long-running opener legitimately holding the lock for longer than openDeadline; extremely slow disk causing repeated IOErr-free but incomplete reads; watchdog restarting bd in a tight loop, each invocation restarting the wait.

Understand the failure class

Related errors

AI-assisted analysis of gastownhall/beads@71377f2769 (2026-08-30). Data as JSON: /api/errors/0a24f9d5ae62d137. Report an issue: GitHub.