kubernetes/kops · error
error fetching GCE managed instance group %q: %v
Error message
error fetching GCE managed instance group %q: %v
What it means
The GCE managed instance group (MIG) named by the node's identity could not be fetched from the Compute Engine API during node identity resolution. kops needs the MIG to walk its instances and map the kubelet instance ID to a GCE instance. It wraps the underlying googleapi error with the MIG name.
Source
Thrown at pkg/nodeidentity/gce/identify.go:237
return instance, nil
}
// getInstanceTemplate queries GCE for the IG Template with the specified name, returning an error if not found
func (i *nodeIdentifier) getInstanceTemplate(name string) (*compute.InstanceTemplate, error) {
t, err := i.computeService.InstanceTemplates.Get(i.project, name).Do()
if err != nil {
return nil, fmt.Errorf("error fetching GCE instance group template %q: %v", name, err)
}
return t, nil
}
// getMIG queries GCE for the MIG with the specified name, returning an error if not found
func (i *nodeIdentifier) getMIG(zone string, migName string) (*compute.InstanceGroupManager, error) {
mig, err := i.computeService.InstanceGroupManagers.Get(i.project, zone, migName).Do()
if err != nil {
return nil, fmt.Errorf("error fetching GCE managed instance group %q: %v", migName, err)
}
return mig, nil
}
// getManagedInstance queries GCE for the instance from the MIG
func (i *nodeIdentifier) getManagedInstance(ctx context.Context, mig *compute.InstanceGroupManager, instanceID uint64) (*compute.ManagedInstance, error) {
var matches []*compute.ManagedInstance
filter := "id=" + strconv.FormatUint(instanceID, 10)
zone := lastComponent(mig.Zone)
if err := i.computeService.InstanceGroupManagers.ListManagedInstances(i.project, zone, mig.Name).Filter(filter).Pages(ctx, func(page *compute.InstanceGroupManagersListManagedInstancesResponse) error {
// Post-filter... filters aren't implemented (b/27605549)
for _, instance := range page.ManagedInstances {
if instance.Id != instanceID {
continue
}
matches = append(matches, instance)View on GitHub (pinned to 4c8573c808)
Solutions
- Confirm the MIG exists in that zone: gcloud compute instance-groups managed describe <name> --zone <zone> --project <project>
- Grant the controller's service account compute.instanceGroupManagers.get (roles/compute.viewer)
- Recreate/replace the node so it belongs to a live MIG (kops rolling-update)
- Retry on transient API failures (429/5xx)
Defensive patterns
Strategy: retry
Validate before calling
migExists, err := computeService.InstanceGroupManagers.Get(project, zone, migName).Do(); migExists != nil // precheck
Type guard
func migValid(m *compute.InstanceGroupManager, err error) bool { return err == nil && m != nil } Try / catch
mig, err := getMIG(zone, name)
if err != nil {
if isNotFound(err) { /* node outlived MIG: delete stale Node */ }
return err
} Prevention
- Never delete MIGs while their Nodes are still registered
- Use kops rolling-update instead of manual MIG deletion
- Verify IAM on the kops-controller service account after IAM changes
When it happens
Trigger: i.computeService.InstanceGroupManagers.Get(project, zone, migName).Do() fails: MIG deleted/renamed, wrong zone, missing compute.instanceGroupManagers.get permission, or API error (403/404/429/5xx).
Common situations: Node outlived its MIG after cluster teardown or migration; zone moved (kops renames instance groups across zones); controller service account IAM stripped; MIG name in providerID/instance metadata stale.
Related errors
- cannot find owner for instance %s
- instance %s did not have Version set
- ig name not set on instance template %s
- error fetching GCE instance: %w
- error fetching GCE instance group template %q: %v
AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05).
Data as JSON: /api/errors/5934e5daf1739bc0.
Report an issue: GitHub.