{"record":{"id":"37bde28e75072ca0","repo":"gastownhall/beads","slug":"hard-timeout-waiting-for-proxy-on-s-safe-child-k","errorCode":null,"errorMessage":"hard timeout waiting for proxy on %s; safe child kill failed: %w","messagePattern":"hard timeout waiting for proxy on (.+?); safe child kill failed: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/storage/dbproxy/proxy/endpoint.go","lineNumber":395,"sourceCode":"\t\t\tif errors.As(childErr, &exitErr) && exitErr.ExitCode() == LockHeldExitCode {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\n\t\t\t\t\t\"proxy child lost the proxy.lock spawn race for %s: %w\",\n\t\t\t\t\trootDir, childErr,\n\t\t\t\t)\n\t\t\t}\n\t\t\tif opts.Port != 0 {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\n\t\t\t\t\t\"proxy child exited before becoming ready on explicitly configured port %d (see %s): %w\",\n\t\t\t\t\topts.Port, opts.LogFilePath, childErr,\n\t\t\t\t)\n\t\t\t}\n\t\t\treturn Endpoint{}, fmt.Errorf(\n\t\t\t\t\"proxy child exited before publishing its OS-assigned port (see %s): %w\",\n\t\t\t\topts.LogFilePath, childErr,\n\t\t\t)\n\t\tcase <-hard.C:\n\t\t\tif err := killSpawnedChild(child); err != nil {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\"hard timeout waiting for proxy on %s; safe child kill failed: %w\", describeSpawnPort(opts.Port), err)\n\t\t\t}\n\t\t\treturn Endpoint{}, fmt.Errorf(\"hard timeout (%s) waiting for proxy on %s\", spawnReadyHardTimeout, describeSpawnPort(opts.Port))\n\t\tcase <-poll.C:\n\t\t}\n\t\tif time.Now().After(deadline) {\n\t\t\tif err := killSpawnedChild(child); err != nil {\n\t\t\t\treturn Endpoint{}, fmt.Errorf(\"timeout waiting for proxy on %s; safe child kill failed: %w\", describeSpawnPort(opts.Port), err)\n\t\t\t}\n\t\t\treturn Endpoint{}, fmt.Errorf(\"timeout waiting for proxy to become ready on %s\", describeSpawnPort(opts.Port))\n\t\t}\n\t}\n}\n\n// describeSpawnPort renders a requested spawn port for wait/timeout\n// messages: 0 is the default OS-assigned path, not a literal \"port 0\".\nfunc describeSpawnPort(port int) string {\n\tif port == 0 {\n\t\treturn \"its OS-assigned port\"","sourceCodeStart":377,"sourceCodeEnd":413,"githubUrl":"https://github.com/gastownhall/beads/blob/71377f276968b452ee607177637970a4ff888584/internal/storage/dbproxy/proxy/endpoint.go#L377-L413","documentation":"Returned by spawnAndHandoff when the hard readiness timer (spawnReadyHardTimeout) fires and the subsequent attempt to safely kill the child process fails. It combines a startup timeout with a process-management failure, so both problems need attention.","triggerScenarios":"The spawned proxy child neither becomes ready, exits, nor hits the caller deadline before spawnReadyHardTimeout elapses, and killSpawnedChild returns an error (e.g. process already reaped, permission denied sending the signal, or handle invalid).","commonSituations":"Extremely slow disk or database init exceeding the hard timeout; system under heavy load starving the child; permission issues (different uid, container restrictions) preventing the parent from killing the child; PID reuse/namespace quirks in containers.","solutions":["Investigate why killSpawnedChild failed (permissions, PID namespace, signal restrictions) — a zombie child may still be running and holding the lock/port.","Check the child log and ps/pgrep for a lingering db-proxy-child process; kill it manually if orphaned.","Free up system resources or increase spawnReadyHardTimeout if startup is legitimately slow.","Re-run the start in a cleaner environment (same user, same PID namespace) if container/permission quirks caused the kill failure."],"exampleFix":"// before: ignoring a failed hard-timeout kill leaves orphan children\nep, err := GetCreateDatabaseProxyServerEndpoint(root, opts)\n// after: on failure, sweep lingering children before retrying\nif err != nil && strings.Contains(err.Error(), \"safe child kill failed\") {\n    exec.Command(\"pkill\", \"-f\", \"db-proxy-child --root \"+root).Run()\n    ep, err = GetCreateDatabaseProxyServerEndpoint(root, opts)\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"ep, err := GetCreateDatabaseProxyServerEndpoint(rootDir, opts)\nif err != nil && strings.Contains(err.Error(), \"safe child kill failed\") {\n    // a child may be orphaned; sweep it before retrying\n    exec.Command(\"pkill\", \"-f\", \"db-proxy-child --root \"+rootDir).Run()\n    time.Sleep(250 * time.Millisecond)\n    ep, err = GetCreateDatabaseProxyServerEndpoint(rootDir, opts)\n}","preventionTips":["Run the parent and child under the same UID so the kill signal is permitted","Avoid PID-namespace splitting between caller and proxy (same container/host)","Watch for lingering db-proxy-child processes after failures","Raise timeouts if the host is slow rather than letting the hard timer fire"],"tags":["go","process-spawn","timeout","process-kill"],"backgroundTag":"process-kill-timeout","analyzedSha":"71377f276968b452ee607177637970a4ff888584","analyzedAt":"2026-08-30T18:55:39.744Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}