kubernetes/kops · error

failed to delete %s: %w

Error message

failed to delete %s: %w

What it means

pruneObjects deletes each live object whose namespace/name key is not in the manifest keep-set. A failed Delete call is wrapped as `failed to delete <namespace/name>`, aborting the prune of that kind at the first failure.

Source

Thrown at channels/pkg/channels/prune.go:144

		namespace := actualObject.GetNamespace()
		key := namespace + "/" + name
		if _, found := keepMap[key]; found {
			// Object is in manifest, don't delete
			continue
		}

		klog.Infof("pruning %s %s", gvr, key)

		var resource dynamic.ResourceInterface
		if namespace != "" {
			resource = p.Client.Resource(gvr).Namespace(namespace)
		} else {
			resource = p.Client.Resource(gvr)
		}

		var opts v1.DeleteOptions
		if err := resource.Delete(ctx, name, opts); err != nil {
			return fmt.Errorf("failed to delete %s: %w", key, err)
		}
	}

	return nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Read the wrapped inner error: 403 → add delete RBAC; 409 → retry; webhook error → fix or remove the blocking webhook
2. Clear stuck finalizers on the object (`kubectl patch <obj> -p '{"metadata":{"finalizers":[]}}' --type=merge`) then re-run prune
3. Delete the object manually with kubectl to confirm the cause, then re-run the channel update
4. Ensure no other controller is fighting over the object during prune

Example fix

// before (object stuck with finalizer)
metadata:
  finalizers:
  - example.com/blocker
// after
kubectl patch myresource -n ns name --type=merge -p '{"metadata":{"finalizers":null}}'
# then re-run kops update

// pre-check delete permission for the pruned kind
ok, _ := authClient.SelfSubjectAccessReviews().Create(ctx, &authorizationv1.SelfSubjectAccessReview{
    Spec: authorizationv1.SelfSubjectAccessReviewSpec{
        ResourceAttributes: &authorizationv1.ResourceAttributes{
            Verb: "delete", Group: gk.Group, Resource: resourceName}},
})
if !ok.Status.Allowed {
    return fmt.Errorf("RBAC denies delete on %s", resourceName)
}

if err := pruner.Prune(ctx, manifest, spec); err != nil {
    var delErr error
    if strings.Contains(err.Error(), "failed to delete ") {
        delErr = errors.Unwrap(err)
        if apierrors.IsConflict(delErr) {
            return retryPrune(err) // 409: transient, safe to retry
        }
        return fmt.Errorf("object stuck (finalizer/webhook?): %w", delErr)
    }
    return err
}

When it happens

Trigger: The dynamic client Delete of a stale object fails: RBAC denies delete, the object has a finalizer blocking deletion (deletion timestamp set but object persists), the object was concurrently modified/removed (409/404), or admission webhooks reject deletion.

Common situations: Objects stuck in Terminating due to orphaned finalizers; missing delete permission for the pruned kind; webhook (e.g. validating admission webhook) unavailable so DELETE is rejected; race with another controller recreating/editing the object.

Related errors

• failed to prune objects of kind %s: %w
• error deleting SSHCredential: %v
• error reading Keyset %q: %v
• DeleteGroup not implemented on azureCloud
• error adding needs-update label: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/f4b59f87355c3740. Report an issue: GitHub.