kubernetes/kops · error

failed to decode user data: %w

Error message

failed to decode user data: %w

What it means

Azure stores VMSS custom data base64-encoded. Find() decodes profile.userData with base64.StdEncoding; if the stored string is not valid base64, decoding fails and kOps wraps the decoder error with this message, since it cannot reconstruct the task's UserData resource.

Source

Thrown at upup/pkg/fi/cloudup/azuretasks/vmscaleset.go:186

	if osProfile.LinuxConfiguration == nil {
		return nil, fmt.Errorf("found VMSS without Linux config")
	}
	if osProfile.LinuxConfiguration.SSH == nil {
		return nil, fmt.Errorf("found VMSS without SSH config")
	}
	if osProfile.LinuxConfiguration.SSH.PublicKeys == nil {
		return nil, fmt.Errorf("found VMSS without SSH public keys")
	}
	sshKeys := osProfile.LinuxConfiguration.SSH.PublicKeys
	if len(sshKeys) != 1 {
		return nil, fmt.Errorf("expecting exactly 1 SSH key for %q, found %d: %+v", *s.Name, len(sshKeys), sshKeys)
	}

	var userData []byte
	if profile.UserData != nil {
		userData, err = base64.StdEncoding.DecodeString(*profile.UserData)
		if err != nil {
			return nil, fmt.Errorf("failed to decode user data: %w", err)
		}
	}

	vmss := &VMScaleSet{
		Name:      s.Name,
		Lifecycle: s.Lifecycle,
		ResourceGroup: &ResourceGroup{
			Name: s.ResourceGroup.Name,
		},
		VirtualNetwork: &VirtualNetwork{
			Name: to.Ptr(subnetID.VirtualNetworkName),
		},
		Subnet: &Subnet{
			ID: ipConfig.Properties.Subnet.ID,
		},
		StorageProfile: &VMScaleSetStorageProfile{
			VirtualMachineScaleSetStorageProfile: profile.StorageProfile,
		},

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-apply the node userdata through kops: `kops update cluster --yes` then `kops rolling-update cluster` so customData is re-encoded correctly.
2. If writing custom data yourself, always encode with standard base64: `base64.StdEncoding.EncodeToString(data)` (or `base64 -w0` in shell).
3. Inspect the raw value (`az vmss show ... --query "virtualMachineProfile.osProfile.userData"`) and check whether it is URL-safe base64; re-encode it.
4. Ensure any external provisioning tool uses the same standard encoding kOps expects.

Example fix

// before (URL-safe / raw)
customData = to.Ptr(base64.URLEncoding.EncodeToString(d))
// after
customData = to.Ptr(base64.StdEncoding.EncodeToString(d))

if _, err := base64.StdEncoding.DecodeString(fi.ValueOf(profile.UserData)); err != nil {
  return fmt.Errorf("VMSS userdata is not valid standard base64: %w", err)
}

data, err := base64.StdEncoding.DecodeString(*profile.UserData)
if err != nil {
  return fmt.Errorf("kops: cannot decode VMSS userdata: %w", err)
}

When it happens

Trigger: Find() on a VMSS whose virtualMachineProfile.osProfile.userData is non-nil but not valid standard base64 (e.g. URL-safe base64, raw text, or a corrupted value written by external tooling).

Common situations: User data written directly via ARM/CLI without base64 encoding; tools that use URL-safe encoding instead of standard; manual edits or migrations corrupting the value.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• error rendering UserData: %s
• expected exactly one subnet for InstanceGroup %q; subnets wa
• unexpected subnet type: for InstanceGroup %q; type was %s
• instance group must have the same min and max size in Azure,
• malformed format of image urn: %s

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/5f2a3ad5fba64e6e. Report an issue: GitHub.