kubernetes/kops · error

error listing managed instances for %q: %v

Error message

error listing managed instances for %q: %v

What it means

Inside DeleteCloudInstanceGroup's polling loop, each iteration calls ListManagedInstances to check remaining VMs in the MIG. If that list call fails, the error is wrapped with the MIG name and returned immediately, aborting the delete instead of continuing to wait.

Source

Thrown at upup/pkg/fi/cloudup/gce/instancegroups.go:62

// DeleteCloudInstanceGroup deletes the InstanceGroupManager and current InstanceTemplate
func DeleteCloudInstanceGroup(c GCECloud, g *cloudinstances.CloudInstanceGroup) error {
	mig := g.Raw.(*compute.InstanceGroupManager)
	err := DeleteMIGInstances(c, mig)
	if err != nil {
		return err
	}

	timeout := time.Now().Add(10 * time.Minute)

	klog.Infof("Waiting for instances in MIG %q to terminate...", mig.Name)
	for {
		if time.Now().After(timeout) {
			return fmt.Errorf("timed out waiting for instances in MIG %q to terminate", mig.Name)
		}

		instances, err := ListManagedInstances(c, mig)
		if err != nil {
			return fmt.Errorf("error listing managed instances for %q: %v", mig.Name, err)
		}
		if len(instances) == 0 {
			klog.Infof("All instances in MIG %q terminated", mig.Name)
			break
		}
		klog.Infof("%d instance(s) remaining in MIG %q, waiting...", len(instances), mig.Name)
		time.Sleep(PollingInterval)
	}

	err = DeleteInstanceGroupManager(c, mig)
	if err != nil {
		return err
	}

	return DeleteInstanceTemplate(c, mig.InstanceTemplate)
}

// DeleteInstance deletes a GCE instance

View on GitHub (pinned to 4c8573c808)

Solutions

1. Inspect the wrapped inner error (`%v`) for the root cause: 403/permission → grant compute API roles to the service account; 404 → the MIG is already gone, safe to proceed.
2. For rate limiting (429/403 quota), retry with backoff or reduce concurrent deletions.
3. Verify the MIG's zone/name are correct and the Compute Engine API is enabled for the project.
4. Re-run `kops delete` after the transient API failure clears.

Example fix

// before: list fails on 404 (MIG already deleted) and aborts
instances, err := ListManagedInstances(c, mig)
// after (caller-level): treat already-deleted MIG as success
if strings.Contains(err.Error(), "notFound") {
    return nil // MIG already deleted
}

// confirm API access before starting deletion loops
_, err := svc.InstanceGroupManagers.List(project, zone).MaxResults(1).Context(ctx).Do()
if err != nil {
	return fmt.Errorf("compute API unreachable or unauthorized: %w", err)
}

err := cloud.DeleteCloudInstanceGroup(mig)
if err != nil && strings.Contains(err.Error(), "error listing managed instances") {
	var gerr *googleapi.Error
	if errors.As(err, &gerr) {
		switch gerr.Code {
		case http.StatusNotFound:
			return nil // already deleted
		case http.StatusForbidden:
			return fmt.Errorf("missing compute API permissions: %w", err)
		default:
			return retryWithBackoff(err)
		}
	}
}

When it happens

Trigger: ListManagedInstances (compute.InstanceGroupManagers.ListInstances) returning a non-nil error during the delete loop — transient API errors (5xx, rate limiting/quota), permission denied on the compute API, invalid MIG name/zone, or the MIG being deleted concurrently so the endpoint 404s mid-loop.

Common situations: kops delete cluster / delete instancegroup on GCE hitting API rate limits with many MIGs, a service account missing compute.viewer rights, regional/zone mismatch, or a concurrently deleted MIG.

Related errors

• error fetching GCE managed instance group %q: %v
• timed out waiting for instances in MIG %q to terminate
• cannot find owner for instance %s
• instance %s did not have Version set
• ig name not set on instance template %s

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/c9577325d4802b58. Report an issue: GitHub.