{"record":{"id":"28cc3384d9858d1d","repo":"k3s-io/k3s","slug":"managed-etcd-cluster-membership-has-been-reset-re","errorCode":null,"errorMessage":"Managed etcd cluster membership has been reset, restart without --cluster-reset flag now. Backup and delete ${datadir}/server/db on each peer etcd server and rejoin the nodes","messagePattern":"Managed etcd cluster membership has been reset, restart without --cluster-reset flag now\\. Backup and delete (.+?)/server/db on each peer etcd server and rejoin the nodes","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"info","filePath":"pkg/etcd/etcd.go","lineNumber":361,"sourceCode":"\t\t\t\tif err != nil {\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\n\t\t\t\tif rebootstrap != nil {\n\t\t\t\t\t// storageBootstrap() - runtime structure has been written with correct certificate data\n\t\t\t\t\tif err := rebootstrap(); err != nil {\n\t\t\t\t\t\tlogrus.Fatal(err)\n\t\t\t\t\t}\n\t\t\t\t}\n\n\t\t\t\t// call functions to rewrite them from daemons/control/server.go (prepare())\n\t\t\t\tif err := deps.GenServerDeps(e.config); err != nil {\n\t\t\t\t\tlogrus.Fatal(err)\n\t\t\t\t}\n\n\t\t\t\tif len(members.Members) == 1 && members.Members[0].Name == e.name {\n\t\t\t\t\t// Cancel the process context so that it will shut down.\n\t\t\t\t\tsignals.RequestShutdown(errors.New(\"Managed etcd cluster membership has been reset, restart without --cluster-reset flag now. Backup and delete ${datadir}/server/db on each peer etcd server and rejoin the nodes\"))\n\t\t\t\t\treturn true, nil\n\t\t\t\t}\n\t\t\t} else if e.client != nil {\n\t\t\t\t// make sure that peer ips are updated to the node ip in case the test fails\n\t\t\t\tmembers, err := e.client.MemberList(ctx)\n\t\t\t\tif err != nil {\n\t\t\t\t\tlogrus.Warnf(\"failed to list etcd members: %v\", err)\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\t\t\t\tif len(members.Members) > 1 {\n\t\t\t\t\tlogrus.Warnf(\"failed to update peer url: etcd still has more than one member\")\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\t\t\t\tif _, err := e.client.MemberUpdate(ctx, members.Members[0].ID, []string{e.peerURL()}); err != nil {\n\t\t\t\t\tlogrus.Warnf(\"failed to update peer url: %v\", err)\n\t\t\t\t\treturn false, nil\n\t\t\t\t}\n\t\t\t}","sourceCodeStart":343,"sourceCodeEnd":379,"githubUrl":"https://github.com/k3s-io/k3s/blob/6ba341e396edc16b8dcae978a7c5e3ac7ee5606e/pkg/etcd/etcd.go#L343-L379","documentation":"Not a failure: after --cluster-reset completes and the member list contains exactly this single member, k3s calls signals.RequestShutdown with this message to stop the process. It instructs the operator to restart without --cluster-reset and to have every peer back up and delete its db directory before rejoining the reset cluster.","triggerScenarios":"Running k3s server --cluster-reset (optionally --cluster-reset-restore-path=<snapshot>) successfully; the completion path detects the single-member condition and deliberately shuts the supervisor down.","commonSituations":"Planned disaster recovery from a snapshot; resetting a quorum-lost cluster; operators surprised that the process exits after reset.","solutions":["Restart k3s without --cluster-reset; the reset node boots as the sole member.","On each former peer: back up and delete /var/lib/rancher/k3s/server/db, then start that server with --server https://<reset-node>:6443 to rejoin.","Verify quorum and workload health after peers rejoin."],"exampleFix":"# after seeing this message on the reset node\nsystemctl restart k3s   # no --cluster-reset\n# on each peer\nmv /var/lib/rancher/k3s/server/db /var/lib/rancher/k3s/server/db.bak\nk3s server --server https://<reset-node>:6443","handlingStrategy":"try-catch","validationCode":"// Automation wrapper: detect completion of a reset before moving on.\nif out, err := run(\"k3s\", \"server\", \"--cluster-reset\"); err != nil {\n\tif strings.Contains(string(out), \"restart without --cluster-reset\") { /* success path */ }\n}","typeGuard":null,"tryCatchPattern":"// The message arrives via process shutdown: catch it in your supervisor/automation.\nif strings.Contains(shutdownReason.Error(), \"Managed etcd cluster membership has been reset\") {\n\t// expected: restart service WITHOUT --cluster-reset; then drive peers to wipe db and rejoin\n}","preventionTips":["Automate the two-step reset: run with --cluster-reset once, detect this shutdown, restart clean.","Keep peer runbooks ready: back up + delete db, rejoin with --server.","Never leave --cluster-reset in the systemd unit across restarts."],"tags":["k3s","etcd","cluster-reset","recovery","shutdown"],"backgroundTag":null,"analyzedSha":"6ba341e396edc16b8dcae978a7c5e3ac7ee5606e","analyzedAt":"2026-08-15T16:27:54.286Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}