kubernetes/kops · error

error applying annotation to record addon installation: %v

Error message

error applying annotation to record addon installation: %v

What it means

updateAddon wraps failures from Channel.SetInstalledVersion, which records the installed addon version as an annotation (addons.k8s.io/<name>) on the target namespace. The addon manifest was applied, but the installed-version annotation could not be written, so subsequent runs will consider the addon out-of-date and re-apply it.

Source

Thrown at channels/pkg/channels/addon.go:232

			merr = multierr.Append(merr, fmt.Errorf("error applying update after prune: %w", err))
		} else {
			// If we succeeded to apply after prune, clear the errors
			merr = nil
		}
	}

	if merr != nil {
		return fmt.Errorf("error updating addon from %q: %w", manifestURL, merr)
	}

	if err := a.AddNeedsUpdateLabel(ctx, k8sClient, required); err != nil {
		return fmt.Errorf("error adding needs-update label: %v", err)
	}

	channel := a.buildChannel()
	err = channel.SetInstalledVersion(ctx, k8sClient, a.ChannelVersion())
	if err != nil {
		return fmt.Errorf("error applying annotation to record addon installation: %v", err)
	}
	return nil
}

func (a *Addon) AddNeedsUpdateLabel(ctx context.Context, k8sClient kubernetes.Interface, required *AddonUpdate) error {
	if required.ExistingVersion != nil {
		if a.Spec.NeedsRollingUpdate != "" {
			err := a.patchNeedsUpdateLabel(ctx, k8sClient)
			if err != nil {
				return fmt.Errorf("error patching needs-update label: %v", err)
			}
		}
	}
	return nil
}

func (a *Addon) patchNeedsUpdateLabel(ctx context.Context, k8sClient kubernetes.Interface) error {
	klog.Infof("addon %v wants to update %v nodes", a.Name, a.Spec.NeedsRollingUpdate)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Ensure the target namespace exists and the caller's RBAC allows get/patch on namespaces.
2. Re-run the apply; recording the version is retried automatically next run since the recorded hash won't match.
3. Check cluster API server health and the kubeconfig context.
4. Clear stale addons.k8s.io/* annotations on the namespace if corrupted, then re-apply.

Example fix

// before
err = channel.SetInstalledVersion(ctx, k8sClient, a.ChannelVersion())
if err != nil {
	return fmt.Errorf("error applying annotation to record addon installation: %v", err)
}
// after
if err := channel.SetInstalledVersion(ctx, k8sClient, a.ChannelVersion()); err != nil {
	return fmt.Errorf("error applying annotation to record addon installation for %q: %w", a.Name, err)
}

_, err := k8sClient.CoreV1().Namespaces().Get(ctx, "kube-system", metav1.GetOptions{})
if err != nil {
	return fmt.Errorf("namespace not accessible, version annotation will fail: %w", err)
}

if err := updateAddon(...); err != nil {
	if strings.Contains(err.Error(), "error applying annotation to record addon installation") {
		// idempotent: re-running will re-record the version annotation
		retryWithBackoff(3, func() error { return updateAddon(...) })
	}
}

When it happens

Trigger: EnsureUpdated -> updateAddon calls SetInstalledVersion after a successful apply; failure occurs when the namespace Get fails (namespace missing/not ready, RBAC), ChannelVersion.Encode() fails, or the namespace strategic-merge Patch fails (RBAC, conflict, API server error).

Common situations: kops applying addons before target namespaces exist; service account lacking patch permission on namespaces; 'kops update cluster' racing with namespace deletion; kubeconfig pointing at the wrong cluster.

Related errors

• error applying annotation to namespace: %v
• error adding needs-update label: %v
• error parsing version spec %q
• error querying namespace %q: %v
• failed to apply objects: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/022b40edd5fd0b31. Report an issue: GitHub.