{"record":{"id":"72c0d0879de55a98","repo":"charmbracelet/crush","slug":"failed-to-create-workspace-server-kept-shutting-d","errorCode":null,"errorMessage":"failed to create workspace: server kept shutting down","messagePattern":"failed to create workspace: server kept shutting down","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/cmd/root.go","lineNumber":481,"sourceCode":"// instead of failing the command.\nfunc createWorkspaceOnLiveServer(\n\tctx context.Context, c *client.Client, req proto.Workspace, replace func() error,\n) (*proto.Workspace, error) {\n\tfor attempt := range maxStaleServerRetries {\n\t\tws, err := c.CreateWorkspace(ctx, req)\n\t\tif err == nil {\n\t\t\treturn ws, nil\n\t\t}\n\t\tif !errors.Is(err, client.ErrServerShuttingDown) || attempt == maxStaleServerRetries-1 {\n\t\t\treturn nil, fmt.Errorf(\"failed to create workspace: %v\", err)\n\t\t}\n\t\tslog.Warn(\"Server is shutting down; retrying against a replacement\",\n\t\t\t\"attempt\", attempt+1, \"error\", err)\n\t\tif err := replace(); err != nil {\n\t\t\treturn nil, err\n\t\t}\n\t}\n\treturn nil, fmt.Errorf(\"failed to create workspace: server kept shutting down\")\n}\n\n// replaceExitingServer waits out the socket of a server that has committed\n// to exiting, then brings up a fresh one.\nfunc replaceExitingServer(cmd *cobra.Command, hostURL *url.URL) error {\n\tif hostURL.Scheme == \"unix\" {\n\t\tif err := awaitSocketGone(cmd.Context(), hostURL); err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\tif err := spawnAndWaitReady(cmd, hostURL); err != nil {\n\t\treturn fmt.Errorf(\"failed to initialize crush server: %v\", err)\n\t}\n\treturn nil\n}\n\n// ensureServer auto-starts a detached server if the socket file does not\n// exist. When the socket exists, it verifies that the running server","sourceCodeStart":463,"sourceCodeEnd":499,"githubUrl":"https://github.com/charmbracelet/crush/blob/7944b8e52225d8805e31eacbf7ef24856b0dfb7a/internal/cmd/root.go#L463-L499","documentation":"createWorkspaceOnLiveServer in internal/cmd/root.go gives up after maxStaleServerRetries attempts when every failure is client.ErrServerShuttingDown, returning the fixed message 'failed to create workspace: server kept shutting down'. It means a dying server kept being hit and even replacement attempts did not yield a healthy server.","triggerScenarios":"Every CreateWorkspace call across all retries returned ErrServerShuttingDown — e.g. multiple stale servers in a shutdown loop, or a supervisor (systemd/docker) continuously restarting/terminating the server on the same socket.","commonSituations":"Container orchestrator repeatedly restarting the crush server; multiple crush processes racing on one unix socket; system shutdown in progress; socket file lingering while its server exits.","solutions":["Find and kill all crush server processes: pgrep -af crush; pkill -f 'crush serve'","Remove the stale socket file and retry so a fresh server spawns","Disable or fix the supervisor (systemd unit / docker restart policy) that keeps cycling the server","Reboot or wait for system shutdown to complete before retrying"],"exampleFix":"// shell, before\ncrush  # keeps hitting a cycling server\n// after\npkill -f 'crush serve'; rm -f ~/.local/share/crush/crush.sock\ncrush","handlingStrategy":"retry","validationCode":"if running, err := serverIsRunning(sock); err == nil && !running {\n    os.Remove(sock) // clear stale socket before connecting\n}","typeGuard":null,"tryCatchPattern":"ws, err := createWorkspaceOnLiveServer(cmd, req, replace)\nif err != nil {\n    if strings.Contains(err.Error(), \"server kept shutting down\") {\n        pkill -f 'crush serve'; os.Remove(sock); // then retry from a clean state\n    }\n    return err\n}","preventionTips":["Kill stale servers and remove stale socket files before connecting","Disable conflicting restart supervisors for the same socket","Investigate why the server keeps exiting (check `crush logs`)"],"tags":["server","workspace","retry","shutdown","socket"],"backgroundTag":"server-shutting-down-loop","analyzedSha":"7944b8e52225d8805e31eacbf7ef24856b0dfb7a","analyzedAt":"2026-08-29T12:48:59.079Z","schemaVersion":2},"datasetVersion":"2026-08-29T17:17:51.833Z"}