{"record":{"id":"b8b58c607f0d7e47","repo":"k3s-io/k3s","slug":"missing-annotation-on-node-s","errorCode":null,"errorMessage":"missing annotation on node %s","messagePattern":"missing annotation on node (.+?)","errorType":"http","errorClass":null,"httpStatus":400,"severity":"error","filePath":"pkg/server/handlers/secrets-encrypt.go","lineNumber":531,"sourceCode":"}\n\nfunc getEncryptionHashAnnotation(core core.Interface) (string, string, error) {\n\tnodeName := os.Getenv(\"NODE_NAME\")\n\tnode, err := core.V1().Node().Get(nodeName, metav1.GetOptions{})\n\tif err != nil {\n\t\treturn \"\", \"\", err\n\t}\n\tif _, ok := node.Labels[util.ControlPlaneRoleLabelKey]; !ok {\n\t\treturn \"\", \"\", fmt.Errorf(\"cannot manage secrets encryption on non control-plane node %s\", nodeName)\n\t}\n\tif ann, ok := node.Annotations[secretsencrypt.EncryptionHashAnnotation]; ok {\n\t\tsplit := strings.Split(ann, \"-\")\n\t\tif len(split) != 2 {\n\t\t\treturn \"\", \"\", fmt.Errorf(\"invalid annotation %s found on node %s\", ann, nodeName)\n\t\t}\n\t\treturn split[0], split[1], nil\n\t}\n\treturn \"\", \"\", fmt.Errorf(\"missing annotation on node %s\", nodeName)\n}\n\n// verifyRotateKeysSupport checks that the k3s version is at least v1.28.0 on all control-plane nodes\nfunc verifyRotateKeysSupport(core core.Interface) error {\n\tlabelSelector := labels.Set{util.ControlPlaneRoleLabelKey: \"true\"}.String()\n\tnodes, err := core.V1().Node().List(metav1.ListOptions{LabelSelector: labelSelector})\n\tif err != nil {\n\t\treturn err\n\t}\n\tfor _, node := range nodes.Items {\n\t\tkubver, err := semver.ParseTolerant(node.Status.NodeInfo.KubeletVersion)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"failed to parse kubelet version %s: %v\", node.Status.NodeInfo.KubeletVersion, err)\n\t\t}\n\t\tsupportVer, err := semver.Make(\"1.28.0\")\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}","sourceCodeStart":513,"sourceCodeEnd":549,"githubUrl":"https://github.com/k3s-io/k3s/blob/6ba341e396edc16b8dcae978a7c5e3ac7ee5606e/pkg/server/handlers/secrets-encrypt.go#L513-L549","documentation":"getEncryptionHashAnnotation requires the local control-plane node to carry the encryption hash annotation. If the Node object has the control-plane label but no annotation at all, the stage/hash cannot be determined and this error is returned instead of guessing.","triggerScenarios":"Any secrets-encrypt operation (status/prepare/rotate stages) where the local Node lacks the EncryptionHashAnnotation: a server that joined after encryption was configured and never ran a stage, a node that was down during the last prepare, or the annotation was deleted.","commonSituations":"Adding a new server to an encrypted cluster and immediately running rotation; a node rejoining after restore without annotations; someone pruning 'unknown' annotations with a cleanup controller.","solutions":["From a node that has the annotation, re-run the current stage ('k3s secrets-encrypt prepare' or the appropriate stage) so all control-plane nodes converge and receive the annotation.","If the whole cluster lost annotations, run 'secrets-encrypt prepare' fresh and complete the subsequent stages in order.","Check for admission controllers or automation that strips annotations and exclude the encryption-hash key.","Confirm the node in question actually runs a current k3s server build (annotation writing requires a version that supports staged rotation)."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Pre-check: every control-plane node has the annotation\nnodes, _ := clientset.CoreV1().Nodes().List(metav1.ListOptions{LabelSelector: \"node-role.kubernetes.io/control-plane=true\"})\nfor _, n := range nodes.Items {\n    if _, ok := n.Annotations[\"encryption.hash\"]; !ok {\n        log.Printf(\"node %s lacks annotation - re-run current stage\", n.Name)\n    }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Complete stages before adding or restarting servers","Re-run the current stage to converge annotations on new/rejoining nodes","Audit for annotation-pruning controllers"],"tags":["annotations","secrets-encryption","cluster-sync"],"backgroundTag":null,"analyzedSha":"6ba341e396edc16b8dcae978a7c5e3ac7ee5606e","analyzedAt":"2026-08-15T16:27:54.286Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}