kubernetes/kops · error

error patching needs-update label: %v

Error message

error patching needs-update label: %v

What it means

AddNeedsUpdateLabel wraps the error from patchNeedsUpdateLabel, which lists nodes matching a role selector and patches a 'kops.k8s.io/needs-update' annotation onto each. It only runs when the addon is an upgrade (ExistingVersion != nil) and Spec.NeedsRollingUpdate is set. Failure means node rolling updates for this addon change will not be triggered.

Source

Thrown at channels/pkg/channels/addon.go:242

	if err := a.AddNeedsUpdateLabel(ctx, k8sClient, required); err != nil {
		return fmt.Errorf("error adding needs-update label: %v", err)
	}

	channel := a.buildChannel()
	err = channel.SetInstalledVersion(ctx, k8sClient, a.ChannelVersion())
	if err != nil {
		return fmt.Errorf("error applying annotation to record addon installation: %v", err)
	}
	return nil
}

func (a *Addon) AddNeedsUpdateLabel(ctx context.Context, k8sClient kubernetes.Interface, required *AddonUpdate) error {
	if required.ExistingVersion != nil {
		if a.Spec.NeedsRollingUpdate != "" {
			err := a.patchNeedsUpdateLabel(ctx, k8sClient)
			if err != nil {
				return fmt.Errorf("error patching needs-update label: %v", err)
			}
		}
	}
	return nil
}

func (a *Addon) patchNeedsUpdateLabel(ctx context.Context, k8sClient kubernetes.Interface) error {
	klog.Infof("addon %v wants to update %v nodes", a.Name, a.Spec.NeedsRollingUpdate)
	selector := ""
	switch a.Spec.NeedsRollingUpdate {
	case api.NeedsRollingUpdateControlPlane:
		selector = "node-role.kubernetes.io/master="
	case api.NeedsRollingUpdateWorkers:
		selector = "node-role.kubernetes.io/node="
	}

	annotationPatch := &annotationPatch{Metadata: annotationPatchMetadata{Annotations: map[string]string{
		"kops.k8s.io/needs-update": "",

View on GitHub (pinned to 4c8573c808)

Solutions

1. Grant the executing identity RBAC permission to list and patch core/v1 Nodes.
2. Retry the operation; the patch loop is idempotent.
3. If nodes were deleted concurrently, rerun after the node group stabilizes.
4. Verify node-role.kubernetes.io labels exist on your nodes for the configured needsRollingUpdate value.

_, err := k8sClient.CoreV1().Nodes().List(ctx, metav1.ListOptions{LabelSelector: "node-role.kubernetes.io/node="})
if err != nil {
	return fmt.Errorf("cannot list nodes for needs-update annotation: %w", err)
}

err := a.AddNeedsUpdateLabel(ctx, k8sClient, required)
if err != nil {
	// safe to retry: strategic-merge patch is idempotent
	retryWithBackoff(3, func() error { return a.AddNeedsUpdateLabel(ctx, k8sClient, required) })
}

When it happens

Trigger: Called from updateAddon during an addon upgrade where the channel spec declares needsRollingUpdate. patchNeedsUpdateLabel fails on: node List with selector 'node-role.kubernetes.io/master=' or '=node' (RBAC, API error), or per-node strategic-merge Patch (node NotFound mid-loop, forbidden).

Common situations: IAM/in-cluster RBAC missing nodes patch rights; addon upgrade that requires node restarts while an autoscaler removes nodes; clusters whose node labels differ from the hard-coded selectors so zero nodes match (no error, but no rolling update).

Related errors

• error adding needs-update label: %v
• error building annotation patch: %v
• error applying annotation to namespace: %v
• applying patch to node: %w
• error applying patch to node: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/8fabee0c2eea3471. Report an issue: GitHub.