{"record":{"id":"f8d52a9860096ff8","repo":"gastownhall/beads","slug":"timeout-s-waiting-for-spawn-marker-s-wait-for","errorCode":null,"errorMessage":"timeout (%s) waiting for spawn marker %s; wait for the in-progress start to finish, then retry","messagePattern":"timeout \\((.+?)\\) waiting for spawn marker (.+?); wait for the in-progress start to finish, then retry","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"internal/storage/dbproxy/proxy/shutdown.go","lineNumber":180,"sourceCode":"\tlockPath := filepath.Join(rootDir, lockName)\n\trecordPath := pidfile.Path(rootDir, pidName)\n\tdeadline := time.Now().Add(shutdownConfirmDeadline)\n\tvar stopped *pidfile.PidFile\n\n\tfor {\n\t\tlock, err := util.TryLock(lockPath)\n\t\tswitch {\n\t\tcase err == nil:\n\t\t\tif checks.CheckSpawnMarker {\n\t\t\t\tactive, markerErr := inspectSpawnMarkerLocked(rootDir)\n\t\t\t\tif markerErr != nil {\n\t\t\t\t\tlock.Unlock()\n\t\t\t\t\treturn nil, markerErr\n\t\t\t\t}\n\t\t\t\tif active {\n\t\t\t\t\tlock.Unlock()\n\t\t\t\t\tif time.Now().After(deadline) {\n\t\t\t\t\t\treturn nil, fmt.Errorf(\n\t\t\t\t\t\t\t\"timeout (%s) waiting for spawn marker %s; wait for the in-progress start to finish, then retry\",\n\t\t\t\t\t\t\tshutdownConfirmDeadline,\n\t\t\t\t\t\t\tfilepath.Join(rootDir, spawnMarkerFileName),\n\t\t\t\t\t\t)\n\t\t\t\t\t}\n\t\t\t\t\ttime.Sleep(shutdownConfirmPoll)\n\t\t\t\t\tcontinue\n\t\t\t\t}\n\t\t\t}\n\n\t\t\tpf, readErr := pidfile.Read(rootDir, pidName)\n\t\t\tif readErr != nil {\n\t\t\t\tlock.Unlock()\n\t\t\t\tif isMalformedPIDFileError(readErr) {\n\t\t\t\t\treturn nil, unverifiableProcessError(\n\t\t\t\t\t\t\"shutdown\",\n\t\t\t\t\t\trecordPath,\n\t\t\t\t\t\t0,","sourceCodeStart":162,"sourceCodeEnd":198,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/internal/storage/dbproxy/proxy/shutdown.go#L162-L198","documentation":"During stopAndAcquire, after acquiring proxy.lock the shutdown path checks the spawn marker: if a start attempt is still in progress (marker active), it unlocks and polls until the marker clears. If the marker remains active past shutdownConfirmDeadline (5s), Shutdown returns this timeout telling the caller a start is mid-flight and to wait and retry. This is intentional backpressure, not a process-management failure.","triggerScenarios":"Calling proxy.Shutdown while a bd start (spawn marked attempt) has held the spawn marker longer than 5 seconds — e.g. a very slow backend start, a hung spawn, or a crash that left a stale spawn marker behind.","commonSituations":"Racing 'bd dolt start' and 'bd dolt stop' from different shells/scripts; backend start slowed by cold cache or heavy IO; a previous start was SIGKILLed leaving an orphaned spawn marker that never clears.","solutions":["Wait for the in-progress start to finish, then re-run the shutdown (as the message states).","Check whether a bd/dolt start process is actually alive (ps aux | grep -E 'bd|dolt'); if it was killed, the marker is stale.","If the marker is stale (no live start), remove <rootDir>/<spawn marker file> and retry the shutdown.","Re-run with backoff in scripts: retry the stop a few times over ~30s before failing.","Avoid racing start/stop in automation — serialize them or wait on the start's exit before stopping."],"exampleFix":"// before (script): immediate stop races start\nbd dolt start &\nbd dolt stop   # timeout (5s) waiting for spawn marker ...\n// after\nbd dolt start &\nwait $! || true\nbd dolt stop","handlingStrategy":"retry","validationCode":"// Before stopping, check whether a start is still marked in progress\nmarker := filepath.Join(rootDir, spawnMarkerFileName)\nif _, err := os.Stat(marker); err == nil {\n    // a start may be in flight; defer the stop or verify no live start process exists\n}","typeGuard":null,"tryCatchPattern":"err := proxy.Shutdown(rootDir)\nfor i := 0; i < 5 && err != nil && strings.Contains(err.Error(), \"waiting for spawn marker\"); i++ {\n    time.Sleep(3 * time.Second)\n    err = proxy.Shutdown(rootDir)\n}\nif err != nil && strings.Contains(err.Error(), \"waiting for spawn marker\") {\n    // likely stale marker: verify no live start process, then remove the marker and retry\n}","preventionTips":["Serialize start/stop operations in scripts; wait for start to exit before stopping","Retry shutdown with backoff when a start is in flight","Treat orphaned spawn markers after SIGKILLed starts: verify no live process, then delete the marker","Keep backend starts fast (warm caches) so the 5s confirm window is not exceeded"],"tags":["go","concurrency","shutdown","race","timeout"],"backgroundTag":"spawn-marker-timeout","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}