kubernetes/kops · error

error resizing InstanceGroupManager %s to 0: %w

Error message

error resizing InstanceGroupManager %s to 0: %w

What it means

This error is returned by DeleteMIGInstances when InstanceGroupManagers().Resize(project, zone, name, 0) fails with anything other than 404 Not Found. Resizing a managed instance group to 0 is how kOps drains all instances from a MIG before deleting it. The googleapi error is wrapped with the MIG selfLink; a missing MIG is treated as success instead.

Source

Thrown at upup/pkg/fi/cloudup/gce/wrappers.go:61

	return c.WaitForOp(op)
}

// DeleteMIGInstances deletes the instances in the MIG in GCE
func DeleteMIGInstances(c GCECloud, t *compute.InstanceGroupManager) error {
	klog.V(2).Infof("Deleting Instances in InstanceGroupManager %s", t.SelfLink)
	u, err := ParseGoogleCloudURL(t.SelfLink)
	if err != nil {
		return err
	}

	op, err := c.Compute().InstanceGroupManagers().Resize(u.Project, u.Zone, u.Name, 0)
	if err != nil {
		if IsNotFound(err) {
			klog.Infof("InstanceGroupManager not found, assuming deleted: %q", t.SelfLink)
			return nil
		}
		return fmt.Errorf("error resizing InstanceGroupManager %s to 0: %w", t.SelfLink, err)
	}

	return c.WaitForOp(op)
}

// DeleteInstanceTemplate deletes the specified InstanceTemplate (by URL) in GCE
func DeleteInstanceTemplate(c GCECloud, selfLink string) error {
	klog.V(2).Infof("Deleting GCE InstanceTemplate %s", selfLink)
	u, err := ParseGoogleCloudURL(selfLink)
	if err != nil {
		return err
	}

	op, err := c.Compute().InstanceTemplates().Delete(u.Project, u.Name)
	if err != nil {
		if IsNotFound(err) {
			klog.Infof("instancetemplate not found, assuming deleted: %q", selfLink)
			return nil

View on GitHub (pinned to 4c8573c808)

Solutions

1. Inspect the wrapped googleapi error for the specific GCE status/operation conflict and wait for the in-flight operation to finish, then retry
2. Disable any autoscaler targeting the MIG (or delete the autoscaler) so it doesn't undo the resize to 0
3. Verify the MIG is zonal; if regional, use the regional resize endpoint or delete via the kOps path appropriate for regional MIGs
4. Check IAM roles (compute.instanceGroupManagerAdmin) for the service account
5. Re-run `kops delete cluster --yes`; the flow is resumable since 404s are skipped

Example fix

// before: assume resize failure means hard stop
if err := DeleteMIGInstances(cloud, igm); err != nil {
	log.Fatalf("teardown failed: %v", err)
}
// after: wait for conflicting op, then retry resize
if err := DeleteMIGInstances(cloud, igm); err != nil {
	if strings.Contains(err.Error(), "already in progress") {
		waitForOpCompletion(cloud, igm.SelfLink)
		err = DeleteMIGInstances(cloud, igm)
	}
	if err != nil {
		log.Fatalf("teardown failed: %v", err)
	}
}

// Ensure the MIG is not autoscaled and has no in-flight op before resizing to 0
autoscalers, err := cloud.Compute().Autoscalers().List(project, zone).Filter(fmt.Sprintf("target eq %s", igm.SelfLink)).Do()
if err != nil {
	return err
}
if len(autoscalers.Items) > 0 {
	return fmt.Errorf("delete autoscaler on %s before resizing to 0", igm.Name)
}

func isGCEConflict(err error) bool {
	var gerr *googleapi.Error
	return errors.As(err, &gerr) && gerr.Code == 409
}

err := gce.DeleteMIGInstances(cloud, igm)
if err != nil {
	var gerr *googleapi.Error
	switch {
	case errors.As(err, &gerr) && gerr.Code == 409:
		// wait for in-flight operation then retry resize
	case errors.As(err, &gerr) && gerr.Code == 404:
		// already gone, proceed
	default:
		return err
	}
}

When it happens

Trigger: Calling DeleteMIGInstances (from DeleteCloudGroup) where the Resize-to-0 call returns a non-NotFound error: resize rejected while another operation on the MIG is in progress, the MIG is regional (zonal resize endpoint mismatch), insufficient IAM permissions, or transient API failures.

Common situations: Cluster teardown where an earlier create/update operation on the MIG is still RUNNING (GCE rejects concurrent ops); autoscaler or another controller re-expanding the group during resize; regional (non-zonal) MIGs that don't accept the zonal resize call; service account missing resize permissions; API 5xx storms during bulk deletes.

Related errors

• error deleting InstanceGroupManager %s: %w
• error listing ManagedInstances in %s: %w
• error deleting Instance %s: %w
• error resizing InstanceGroupManager: %v
• error deleting InstanceTemplate %s: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/d038ecbfce1181ea. Report an issue: GitHub.