kubernetes/kops · error

error deleting old AutoscalingGroup tags: %v

Error message

error deleting old AutoscalingGroup tags: %v

What it means

kops wraps the AWS AutoScaling DeleteTags API error when removing stale tags from an AutoScalingGroup during an update in RenderAWS. This happens after the ASG update succeeded but cleanup of tags that are no longer desired failed.

Source

Thrown at upup/pkg/fi/cloudup/awstasks/autoscalinggroup.go:666

		if changes.CapacityRebalance != nil {
			request.CapacityRebalance = e.CapacityRebalance
			changes.CapacityRebalance = nil
		}

		empty := &AutoscalingGroup{}
		if !reflect.DeepEqual(empty, changes) {
			klog.Warningf("cannot apply changes to AutoScalingGroup: %v", changes)
		}

		klog.V(2).Infof("Updating autoscaling group %s", fi.ValueOf(e.Name))

		if _, err := t.Cloud.Autoscaling().UpdateAutoScalingGroup(ctx, request); err != nil {
			return fmt.Errorf("error updating AutoscalingGroup: %v", err)
		}

		if deleteTagsRequest != nil && len(deleteTagsRequest.Tags) > 0 {
			if _, err := t.Cloud.Autoscaling().DeleteTags(ctx, deleteTagsRequest); err != nil {
				return fmt.Errorf("error deleting old AutoscalingGroup tags: %v", err)
			}
		}
		if updateTagsRequest != nil {
			if _, err := t.Cloud.Autoscaling().CreateOrUpdateTags(ctx, updateTagsRequest); err != nil {
				return fmt.Errorf("error updating AutoscalingGroup tags: %v", err)
			}
		}

		if detachLBRequest != nil {
			if _, err := t.Cloud.Autoscaling().DetachLoadBalancers(ctx, detachLBRequest); err != nil {
				return fmt.Errorf("error detatching LoadBalancers: %v", err)
			}
		}
		if attachLBRequest != nil {
			if _, err := t.Cloud.Autoscaling().AttachLoadBalancers(ctx, attachLBRequest); err != nil {
				return fmt.Errorf("error attaching LoadBalancers: %v", err)
			}
		}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-run `kops update cluster --yes` so the delete is retried (the ASG itself was already updated; only tags are stale).
2. Check IAM policy includes autoscaling:DeleteTags for the kops credentials.
3. If throttled, retry after backoff or reduce concurrent instance-group updates.
4. Compare current ASG tags (`aws autoscaling describe-tags`) with the desired spec to spot manual drift, then let kops reconcile.

// ensure permissions before apply
iam.EnsurePolicy(ctx, "autoscaling:DeleteTags")
// detect tag drift early
aws autoscaling describe-tags --filters Name=resource-id,Values=<asg-name> --region <region>

if _, err := svc.DeleteTags(ctx, req); err != nil {
  if isNotFoundOrThrottle(err) {
    return retryWithBackoff(req)
  }
  return fmt.Errorf("tag cleanup failed (ASG update already applied): %w", err)
}

When it happens

Trigger: During ASG update, a non-empty deleteTagsRequest is built (tags present on the ASG that are absent from the desired spec) and Autoscaling().DeleteTags fails: tag/resource not found, throttling, or missing autoscaling:DeleteTags IAM permission.

Common situations: Tag was already removed out-of-band so the resource version conflicts; role tag renaming (e.g. master to control-plane) causing bulk tag deletions; AWS throttling; credentials lacking DeleteTags permission.

Related errors

• error updating AutoscalingGroup tags: %v
• provider ID cannot be empty
• error updating AutoscalingGroup: %v
• error detatching LoadBalancers: %v
• error attaching LoadBalancers: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/7813539082153633. Report an issue: GitHub.