{"record":{"id":"76e52ce4927c5734","repo":"k3s-io/k3s","slug":"node-name-annotation-for-node-s-not-found","errorCode":null,"errorMessage":"node name annotation for node %s not found","messagePattern":"node name annotation for node (.+?) not found","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/etcd/member_controller.go","lineNumber":55,"sourceCode":"\tnodeController controllerv1.NodeController\n\tctx            context.Context\n}\n\nfunc (e *etcdMemberHandler) sync(key string, node *v1.Node) (*v1.Node, error) {\n\tif node == nil {\n\t\treturn nil, nil\n\t}\n\n\tif _, ok := node.Labels[util.ETCDRoleLabelKey]; !ok {\n\t\tlogrus.Debugf(\"Node %s was not labeled etcd node, skipping sync\", key)\n\t\treturn node, nil\n\t}\n\n\tif removalRequested, ok := node.Annotations[removalAnnotation]; ok {\n\t\t// removal requires node name and address annotations; fail if either are not found\n\t\tname, ok := node.Annotations[NodeNameAnnotation]\n\t\tif !ok {\n\t\t\treturn node, fmt.Errorf(\"node name annotation for node %s not found\", key)\n\t\t}\n\t\taddress, ok := node.Annotations[NodeAddressAnnotation]\n\t\tif !ok {\n\t\t\treturn node, fmt.Errorf(\"node address annotation for node %s not found\", key)\n\t\t}\n\t\tlf := logrus.Fields{\"name\": name, \"address\": address}\n\n\t\tpatch := util.NewPatchList()\n\t\tpatcher := util.NewPatcher[*v1.Node](e.nodeController)\n\n\t\t// Check to see if the node was previously removed from the cluster\n\t\tif removed, ok := node.Annotations[removedNodeNameAnnotation]; ok {\n\t\t\tif removed != name {\n\t\t\t\t// If the current node name is not the same as the removed node name, clear the removal annotations,\n\t\t\t\t// as this indicates that the node has been re-added with a new name.\n\t\t\t\tlogrus.WithFields(lf).Info(\"Resetting removed node flag as removed node name does not match current node name\")\n\t\t\t\tpatch.Remove(\"metadata\", \"annotations\", removedNodeNameAnnotation)\n\t\t\t\tpatch.Remove(\"metadata\", \"annotations\", removalAnnotation)","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/k3s-io/k3s/blob/6ba341e396edc16b8dcae978a7c5e3ac7ee5606e/pkg/etcd/member_controller.go#L37-L73","documentation":"The etcd member controller watches nodes with the etcd role label; when a node carries the removal annotation (etcd.k3s.cattle.io/remove), removal requires both etcd.k3s.cattle.io/node-name and etcd.k3s.cattle.io/node-address annotations, which are normally written by the metadata controller of a healthy etcd member. A removal request without the node-name annotation fails here and the controller will retry.","triggerScenarios":"`kubectl annotate node <n> etcd.k3s.cattle.io/remove=true` on a node labeled node-role.kubernetes.io/etcd=true while the etcd.k3s.cattle.io/node-name annotation is absent (pkg/etcd/member_controller.go:51-56) - the node never registered its member metadata, or annotations were stripped/edited.","commonSituations":"Manually requesting removal of a node that crashed before the metadata controller annotated it; annotations deleted during cleanup or by a mutating webhook; stale remove annotation from a previous migration.","solutions":["Inspect annotations: `kubectl get node <n> -o yaml` - confirm etcd.k3s.cattle.io/node-name / node-address are missing while the remove annotation is present.","If removal is not intended, drop the request: `kubectl annotate node <n> etcd.k3s.cattle.io/remove-`.","If removal is intended and the node is gone, supply the missing annotations yourself from the etcd member list (`etcdctl member list` name) so the controller can proceed, then let it clean up.","If the node is still alive and should be a member, remove the remove annotation and restart its k3s server so the metadata controller republishes node-name/node-address."],"exampleFix":"# before\nkubectl annotate node bad-node etcd.k3s.cattle.io/remove=true\n# -> node name annotation for node bad-node not found\n\n# after (provide the metadata the controller needs, taken from etcdctl member list)\nkubectl annotate node bad-node \\\n  etcd.k3s.cattle.io/node-name=bad-node \\\n  etcd.k3s.cattle.io/node-address=10.0.0.7","handlingStrategy":"validation","validationCode":"// Before requesting etcd node removal, require the metadata annotations:\nnode, _ := clientset.CoreV1().Nodes().Get(ctx, name, metav1.GetOptions{})\nif _, hasRemove := node.Annotations[\"etcd.k3s.cattle.io/remove\"]; hasRemove {\n    if node.Annotations[\"etcd.k3s.cattle.io/node-name\"] == \"\" ||\n        node.Annotations[\"etcd.k3s.cattle.io/node-address\"] == \"\" {\n        log.Fatal(\"cannot request removal: node is missing etcd node-name/node-address annotations\")\n    }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Confirm `kubectl get node <n> -o yaml` shows etcd.k3s.cattle.io/node-name and node-address before adding the remove annotation.","Remove the remove annotation (`kubectl annotate node <n> etcd.k3s.cattle.io/remove-`) if the request was premature.","Let healthy members run long enough for the metadata controller to publish annotations before decommissioning them."],"tags":["etcd","annotations","node-management","kubernetes","k3s"],"backgroundTag":null,"analyzedSha":"6ba341e396edc16b8dcae978a7c5e3ac7ee5606e","analyzedAt":"2026-08-15T16:27:54.286Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}