{"record":{"id":"33a36576c959bf22","repo":"gastownhall/beads","slug":"hard-timeout-s-waiting-for-proxy-on-s","errorCode":null,"errorMessage":"hard timeout (%s) waiting for proxy on %s","messagePattern":"hard timeout \\((.+?)\\) waiting for proxy on (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/storage/dbproxy/proxy/endpoint.go","lineNumber":397,"sourceCode":"\t\t\t\t\t\"proxy child lost the proxy.lock spawn race for %s: %w\",\n\t\t\t\t\trootDir, childErr,\n\t\t\t\t)\n\t\t\t}\n\t\t\tif opts.Port != 0 {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\n\t\t\t\t\t\"proxy child exited before becoming ready on explicitly configured port %d (see %s): %w\",\n\t\t\t\t\topts.Port, opts.LogFilePath, childErr,\n\t\t\t\t)\n\t\t\t}\n\t\t\treturn Endpoint{}, fmt.Errorf(\n\t\t\t\t\"proxy child exited before publishing its OS-assigned port (see %s): %w\",\n\t\t\t\topts.LogFilePath, childErr,\n\t\t\t)\n\t\tcase <-hard.C:\n\t\t\tif err := killSpawnedChild(child); err != nil {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\"hard timeout waiting for proxy on %s; safe child kill failed: %w\", describeSpawnPort(opts.Port), err)\n\t\t\t}\n\t\t\treturn Endpoint{}, fmt.Errorf(\"hard timeout (%s) waiting for proxy on %s\", spawnReadyHardTimeout, describeSpawnPort(opts.Port))\n\t\tcase <-poll.C:\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tif err := killSpawnedChild(child); err != nil {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\"timeout waiting for proxy on %s; safe child kill failed: %w\", describeSpawnPort(opts.Port), err)\n\t\t\t}\n\t\t\treturn Endpoint{}, fmt.Errorf(\"timeout waiting for proxy to become ready on %s\", describeSpawnPort(opts.Port))\n\t\t}\n\t}\n}\n\n// describeSpawnPort renders a requested spawn port for wait/timeout\n// messages: 0 is the default OS-assigned path, not a literal \"port 0\".\nfunc describeSpawnPort(port int) string {\n\tif port == 0 {\n\t\treturn \"its OS-assigned port\"\n\t}\n\treturn fmt.Sprintf(\"port %d\", port)","sourceCodeStart":379,"sourceCodeEnd":415,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/internal/storage/dbproxy/proxy/endpoint.go#L379-L415","documentation":"Returned by spawnAndHandoff when the hard readiness timer (spawnReadyHardTimeout) expires before the proxy child became ready, exited, or the caller's deadline passed — and the child was killed successfully. It reports the configured hard timeout value and the target port description.","triggerScenarios":"GetCreateDatabaseProxyServerEndpoint's child takes longer than spawnReadyHardTimeout to publish a dialable endpoint: slow backend start, blocked bind, wedged filesystem, or a child hanging before it can exit.","commonSituations":"Cold start on a very large Dolt database; NFS/slow workspace storage; host under heavy CPU load; a hung dolt subprocess inside the child; security software (AV) delaying process startup.","solutions":["Check the child log at opts.LogFilePath and any partial discovery record for how far startup got.","Retry once — transient slowness under load often resolves on a warm start.","If startup is legitimately slow, increase the hard timeout constant (spawnReadyHardTimeout) or raise the caller deadline.","Rule out environmental slowdowns (NFS latency, AV scanning, resource limits) on the host."],"exampleFix":"// before: huge database cold start exceeds the fixed hard timeout\nep, err := GetCreateDatabaseProxyServerEndpoint(bigRoot, opts) // hard timeout fires\n// after: pre-warm/start the backend ahead of time, or rely on an already-running proxy\nif existing, derr := DiscoverDatabaseProxyServerEndpoint(bigRoot); derr == nil {\n    ep = existing\n} else {\n    ep, err = GetCreateDatabaseProxyServerEndpoint(bigRoot, opts)\n}","handlingStrategy":"retry","validationCode":"// Estimate feasibility: ensure workspace database exists and disk isn't thrashing\nif _, err := os.Stat(filepath.Join(rootDir, \".dolt\")); err != nil {\n    return fmt.Errorf(\"no dolt database at %s: %w\", rootDir, err)\n}","typeGuard":null,"tryCatchPattern":"ep, err := GetCreateDatabaseProxyServerEndpoint(rootDir, opts)\nif err != nil && strings.Contains(err.Error(), \"hard timeout\") {\n    time.Sleep(time.Second) // let the killed child fully release the lock/port\n    ep, err = GetCreateDatabaseProxyServerEndpoint(rootDir, opts)\n}","preventionTips":["Prefer adopting an already-running proxy over cold starts","Account for large-database cold-start time before spawning","Avoid NFS/AV-scan-protected paths for workspaces","If cold starts legitimately exceed the hard timeout, file to raise spawnReadyHardTimeout"],"tags":["go","process-spawn","timeout","startup-failure"],"backgroundTag":"startup-timeout","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}