{"record":{"id":"c6533946817fd6df","repo":"kubernetes/kops","slug":"nodeup-config-hash-mismatch-was-q-expected-q","errorCode":null,"errorMessage":"nodeup config hash mismatch (was %q, expected %q)","messagePattern":"nodeup config hash mismatch \\(was %q, expected %q\\)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"upup/pkg/fi/nodeup/command.go","lineNumber":163,"sourceCode":"\tcase bootConfig.InstanceGroupName != \"\":\n\t\tnodeupConfigLocation := configBase.Join(\"igconfig\", bootConfig.InstanceGroupRole.ToLowerString(), bootConfig.InstanceGroupName, \"nodeupconfig.yaml\")\n\n\t\tb, err := nodeupConfigLocation.ReadFile(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error loading NodeupConfig %q: %v\", nodeupConfigLocation, err)\n\t\t}\n\n\t\tif err = utils.YamlUnmarshal(b, &nodeupConfig); err != nil {\n\t\t\treturn fmt.Errorf(\"error parsing NodeupConfig %q: %v\", nodeupConfigLocation, err)\n\t\t}\n\t\tnodeupConfigHash = sha256.Sum256(b)\n\tdefault:\n\t\treturn fmt.Errorf(\"no instance group defined in nodeup config\")\n\t}\n\n\tif bootConfig.NodeupConfigHash != \"\" {\n\t\tif want, got := bootConfig.NodeupConfigHash, base64.StdEncoding.EncodeToString(nodeupConfigHash[:]); got != want {\n\t\t\treturn fmt.Errorf(\"nodeup config hash mismatch (was %q, expected %q)\", got, want)\n\t\t}\n\t}\n\n\terr = evaluateSpec(&nodeupConfig, bootConfig.CloudProvider, region)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tarchitecture, err := architectures.FindArchitecture()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error determining OS architecture: %v\", err)\n\t}\n\n\tdistribution, err := distributions.FindDistribution(\"/\")\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error determining OS distribution: %v\", err)\n\t}\n","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/nodeup/command.go#L145-L181","documentation":"After loading nodeupconfig.yaml, nodeup computes its SHA-256 hash and compares it to the base64-encoded NodeupConfigHash embedded in the node's BootConfig. This is an integrity/consistency check ensuring the node boots with the exact config that was provisioned for it. A mismatch means the file in the state store changed (or the boot config is stale) since userdata was generated.","triggerScenarios":"Running NodeUpCommand.Run() when bootConfig.NodeupConfigHash is non-empty and the hash of the freshly-read nodeupconfig.yaml differs from it — e.g. the state store's nodeupconfig.yaml was regenerated by a new 'kops update cluster' while the node still boots with old userdata, or the file was edited/replaced manually.","commonSituations":"Rolling updates in progress: userdata from a previous kops apply plus new config in S3; ASG launch template not refreshed after 'kops update cluster --yes'; manual edits to nodeupconfig.yaml; object-store versioning serving a different object version.","solutions":["Re-run 'kops update cluster --yes' (and 'kops rolling-update cluster --yes') so userdata/launch templates are regenerated in sync with the current nodeupconfig.yaml, then replace the instance.","If the config should not have changed, restore the original nodeupconfig.yaml that matches the boot config hash (from object-store versioning or backup).","Ensure the ASG/instance group launches instances with the latest userdata (refresh instance refresh / launch template version).","Verify no manual edits were made to igconfig/<role>/<ig>/nodeupconfig.yaml in the state store; edit the kOps cluster spec and re-apply instead."],"exampleFix":"// before: node booting with stale userdata after config change\n// instance launches with old launch template\n// after: refresh instance group so userdata hash matches config\nkops update cluster --yes && kops rolling-update cluster --yes","handlingStrategy":"validation","validationCode":"// Verify the config hash matches before running nodeup\nb, _ := nodeupConfigLocation.ReadFile(ctx)\ngot := base64.StdEncoding.EncodeToString(sha256Sum(b))\nif bootConfig.NodeupConfigHash != \"\" && got != bootConfig.NodeupConfigHash {\n    return fmt.Errorf(\"stale boot config: re-run kops update cluster / refresh userdata\")\n}","typeGuard":null,"tryCatchPattern":"err := cmd.Run(out)\nif err != nil && strings.Contains(err.Error(), \"nodeup config hash mismatch\") {\n    // trigger ASG instance refresh so userdata matches current config\n}","preventionTips":["Always run 'kops rolling-update cluster' after 'kops update cluster --yes' so nodes reboot with matching userdata","Enable instance refresh on ASGs / managed instance groups after config changes","Avoid manual edits to state-store files; rely on kOps to keep hashes consistent"],"tags":["nodeup","hash-mismatch","integrity-check","bootstrap"],"backgroundTag":"config-hash-mismatch","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}