{"record":{"id":"543c62f181f39c32","repo":"kubernetes/kops","slug":"error-decoding-ec2-userdata-v","errorCode":null,"errorMessage":"error decoding EC2 UserData: %v","messagePattern":"error decoding EC2 UserData: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"upup/pkg/fi/cloudup/awstasks/instance.go","lineNumber":133,"sourceCode":"\t\tPrivateIPAddress: i.PrivateIpAddress,\n\t\tInstanceType:     i.InstanceType,\n\t\tImageID:          i.ImageId,\n\t\tName:             findNameTag(i.Tags),\n\t}\n\n\t// Fetch instance UserData\n\t{\n\t\trequest := &ec2.DescribeInstanceAttributeInput{}\n\t\trequest.InstanceId = i.InstanceId\n\t\trequest.Attribute = ec2types.InstanceAttributeNameUserData\n\t\tresponse, err := cloud.EC2().DescribeInstanceAttribute(ctx, request)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"error querying EC2 for user metadata for instance %q: %v\", *i.InstanceId, err)\n\t\t}\n\t\tif response.UserData != nil {\n\t\t\tb, err := base64.StdEncoding.DecodeString(aws.ToString(response.UserData.Value))\n\t\t\tif err != nil {\n\t\t\t\treturn nil, fmt.Errorf(\"error decoding EC2 UserData: %v\", err)\n\t\t\t}\n\t\t\tactual.UserData = fi.NewBytesResource(b)\n\t\t}\n\t}\n\n\tif i.SubnetId != nil {\n\t\tactual.Subnet = &Subnet{ID: i.SubnetId}\n\t}\n\tif i.KeyName != nil {\n\t\tactual.SSHKey = &SSHKey{Name: i.KeyName}\n\t}\n\n\tfor _, sg := range i.SecurityGroups {\n\t\tactual.SecurityGroups = append(actual.SecurityGroups, &SecurityGroup{ID: sg.GroupId})\n\t}\n\n\tassociatePublicIpAddress := false\n\tfor _, ni := range i.NetworkInterfaces {","sourceCodeStart":115,"sourceCodeEnd":151,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/cloudup/awstasks/instance.go#L115-L151","documentation":"During Instance.Find, kOps fetches the EC2 instance's UserData attribute via DescribeInstanceAttribute and base64-decodes the returned value. If base64.StdEncoding.DecodeString fails, the task cannot build an accurate 'actual' state and Find aborts with this wrapped error. It indicates the value AWS returned for UserData.Value was not valid base64, which normally points to a corrupted or non-base64-encoded UserData payload stored on the instance.","triggerScenarios":"Instance.Find on an existing EC2 instance whose DescribeInstanceAttribute(UserData) response contains a UserData.Value string that fails base64 decoding (malformed characters, wrong padding, or a raw/JSON-encoded string stored outside kOps).","commonSituations":"Instances imported or mutated outside kOps (e.g. created by terraform/CloudFormation/CLI with UserData set differently), UserData edited by automation that wrote plain text or double-encoded data, stale snapshots of instance attributes, or SDK/region mismatches returning unexpected attribute payloads.","solutions":["Inspect the instance's UserData with `aws ec2 get-instance-attribute --attribute userData --instance-id <id>` and verify it is valid base64 (`echo <value> | base64 -d`).","Re-apply the UserData through kOps (delete/recreate the instance or fix the task spec) so AWS stores a properly encoded value.","If the instance is externally managed, mark it as Shared so kOps does not diff its UserData.","Check for tools that rewrote the attribute and disable them; then rerun `kops update cluster`.","Upgrade kOps/SDK if a version bug in attribute decoding is suspected."],"exampleFix":"// before\nb, err := base64.StdEncoding.DecodeString(aws.ToString(response.UserData.Value))\nif err != nil {\n\treturn nil, fmt.Errorf(\"error decoding EC2 UserData: %v\", err)\n}\n// after\nraw := aws.ToString(response.UserData.Value)\nb, err := base64.StdEncoding.DecodeString(raw)\nif err != nil {\n\t// tolerate externally-managed (non-base64) userdata instead of failing Find\n\tklog.V(2).Infof(\"ignoring non-base64 UserData for instance: %v\", err)\n} else {\n\tactual.UserData = fi.NewBytesResource(b)\n}","handlingStrategy":"validation","validationCode":"// Validate base64 decodability before relying on kOps Find / stored UserData\nimport \"encoding/base64\"\n\nfunc isValidBase64(s string) bool {\n\t_, err := base64.StdEncoding.DecodeString(s)\n\treturn err == nil\n}\n\nraw, _ := ec2Client.DescribeInstanceAttribute(ctx, &ec2.DescribeInstanceAttributeInput{\n\tInstanceId: aws.String(instID), Attribute: ec2types.InstanceAttributeNameUserData,\n})\nif raw.UserData != nil && !isValidBase64(aws.ToString(raw.UserData.Value)) {\n\t// fix or exclude this instance before running kops update\n}","typeGuard":"func hasDecodableUserData(v *string) bool {\n\tif v == nil {\n\t\treturn true\n\t}\n\t_, err := base64.StdEncoding.DecodeString(*v)\n\treturn err == nil\n}","tryCatchPattern":"actual, err := instanceTask.Find(ctx, cloud)\nif err != nil {\n\tif strings.Contains(err.Error(), \"error decoding EC2 UserData\") {\n\t\t// treat as externally managed / re-apply userdata\n\t\tklog.Warningf(\"skipping instance with undecodable userdata: %v\", err)\n\t\treturn nil\n\t}\n\treturn err\n}","preventionTips":["Always set EC2 UserData through kOps or with base64 encoding (--user-data file://... encodes automatically via the CLI).","Don't let external automation rewrite instance attributes of kOps-managed instances.","Mark externally managed instances as shared:true so Find/Render don't diff their UserData.","Spot-check with `aws ec2 get-instance-attribute --attribute userData` before kops update after import."],"tags":["aws","ec2","base64","userdata","instance-find"],"backgroundTag":"invalid-base64-data","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}