kubernetes/kops · error

error applying patch to node: %v

Error message

error applying patch to node: %v

What it means

patchNodeLabels sends the marshaled strategic-merge patch to the Node object; this wraps the Nodes().Patch API call failing, e.g. concurrent node modification (conflict), RBAC denial, or the node having been deleted mid-reconcile.

Source

Thrown at cmd/kops-controller/controllers/node_controller.go:168

		nodePatchMetadata.Labels[k] = &v
	}
	for k := range deleteLabels {
		nodePatchMetadata.Labels[k] = nil
	}

	nodePatch := &nodePatch{
		Metadata: nodePatchMetadata,
	}
	nodePatchJson, err := json.Marshal(nodePatch)
	if err != nil {
		return fmt.Errorf("error building node patch: %v", err)
	}

	klog.V(2).Infof("sending patch for node %q: %q", node.Name, string(nodePatchJson))

	_, err = client.Nodes().Patch(ctx, node.Name, types.StrategicMergePatchType, nodePatchJson, metav1.PatchOptions{})
	if err != nil {
		return fmt.Errorf("error applying patch to node: %v", err)
	}

	return nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Inspect the wrapped error for the HTTP status/reason
2. Check RBAC: kops-controller needs 'patch' on nodes
3. If conflict (409), rely on controller-runtime requeue — it retries automatically
4. If 404, the node is gone; treat as benign
5. Verify API server connectivity from the controller pod

Example fix

// before
_, err = client.Nodes().Patch(ctx, node.Name, types.StrategicMergePatchType, nodePatchJson, metav1.PatchOptions{})
// after: retry on conflict handled by controller-runtime — ensure Requeue is returned
if apierrors.IsConflict(err) {
	return ctrl.Result{Requeue: true}, nil
}

_, err = client.Nodes().Patch(ctx, node.Name, types.StrategicMergePatchType, nodePatchJson, metav1.PatchOptions{})
if err != nil {
	if apierrors.IsConflict(err) || apierrors.IsNotFound(err) {
		return nil // requeue/skip: benign
	}
	return fmt.Errorf("error applying patch to node: %v", err)
}

When it happens

Trigger: The Patch request fails — HTTP 409 conflict (Node updated concurrently / resourceVersion change), 404 (Node deleted), 401/403 (RBAC denies nodes/status patch), or transient network/server errors.

Common situations: kops-controller ServiceAccount lacking patch permission on nodes, node deleted during reconciliation (common with autoscaling churn), heavy concurrent label updates (conflict), or API server temporarily unreachable.

Related errors

• error applying annotation to namespace: %v
• applying patch to node: %w
• error adding needs-update label: %v
• error applying annotation to record addon installation: %v
• error patching needs-update label: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/7ada03c0abbabf66. Report an issue: GitHub.