{"record":{"id":"05ed76f49d288f48","repo":"kubernetes/kops","slug":"error-recreating-instance-s-v","errorCode":null,"errorMessage":"error recreating Instance %s: %v","messagePattern":"error recreating Instance (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"upup/pkg/fi/cloudup/gce/instancegroups.go","lineNumber":113,"sourceCode":"\n// recreateCloudInstance recreates the specified instances, managed by an InstanceGroupManager\nfunc recreateCloudInstance(c GCECloud, i *cloudinstances.CloudInstance) error {\n\tmig := i.CloudInstanceGroup.Raw.(*compute.InstanceGroupManager)\n\n\tklog.V(2).Infof(\"Recreating GCE Instance %s in MIG %s\", i.ID, mig.Name)\n\n\tmigURL, err := ParseGoogleCloudURL(mig.SelfLink)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\top, err := c.Compute().InstanceGroupManagers().RecreateInstances(migURL.Project, migURL.Zone, migURL.Name, i.ID)\n\tif err != nil {\n\t\tif IsNotFound(err) {\n\t\t\tklog.Infof(\"Instance not found, assuming deleted: %q\", i.ID)\n\t\t\treturn nil\n\t\t}\n\t\treturn fmt.Errorf(\"error recreating Instance %s: %v\", i.ID, err)\n\t}\n\n\treturn c.WaitForOp(op)\n}\n\n// GetCloudGroups returns a map of CloudGroup that backs a list of instance groups\nfunc (c *gceCloudImplementation) GetCloudGroups(cluster *kops.Cluster, instancegroups []*kops.InstanceGroup, warnUnmatched bool, nodes []v1.Node) (map[string]*cloudinstances.CloudInstanceGroup, error) {\n\treturn GetCloudGroups(c, cluster, instancegroups, warnUnmatched, nodes)\n}\n\nfunc GetCloudGroups(c GCECloud, cluster *kops.Cluster, instancegroups []*kops.InstanceGroup, warnUnmatched bool, nodes []v1.Node) (map[string]*cloudinstances.CloudInstanceGroup, error) {\n\tgroups := make(map[string]*cloudinstances.CloudInstanceGroup)\n\n\tproject := c.Project()\n\tctx := context.Background()\n\n\tnodesByProviderID := make(map[string]*v1.Node)\n","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/cloudup/gce/instancegroups.go#L95-L131","documentation":"This error is returned by recreateCloudInstance when the GCE Compute API call InstanceGroupManagers().RecreateInstances() fails for a MIG member instance. kOps issues a recreateInstances request on the managed instance group to recreate a specific instance (e.g. during a rolling replace); any API error other than not-found is wrapped with the instance ID and the underlying cause. A not-found error is deliberately swallowed and treated as already deleted.","triggerScenarios":"Calling DeleteInstance -> recreateCloudInstance for an instance whose ID yields a RecreateInstances API error: MIG not found at migURL (project/zone/name), instance not a member of the MIG, GCE quota/regional outage, invalid or stale instance ID, or permission denied on the compute.instanceGroupManagers.updateInstances permission.","commonSituations":"Rolling-update/replace of a GCE cluster where the instance was concurrently deleted, the MIG was removed out-of-band, the service account lacks compute admin IAM roles, or a transient GCE API failure occurs during `kops rolling-update cluster`.","solutions":["Read the wrapped %v cause: if it's a GCE 4xx about the MIG or instance, verify the instance still exists and belongs to the MIG (gcloud compute instance-groups managed describe).","Check IAM: ensure the kOps service account has roles/compute.instanceAdmin.v1 (recreateInstances permission).","If the instance was already deleted externally, re-run; kOps may now see it gone and skip recreation.","Retry on transient Google API errors (5xx / rate limits); rolling-update commands are typically safe to re-run.","Confirm project/zone/name in the MIG URL match the cluster's cloud config."],"exampleFix":"// before\nreturn fmt.Errorf(\"error recreating Instance %s: %v\", i.ID, err)\n// after (diagnose the cause before retrying)\nif gerr, ok := err.(*googleapi.Error); ok && gerr.Code == 404 {\n    klog.Infof(\"MIG or instance no longer exists: %q\", i.ID)\n    return nil\n}\nreturn fmt.Errorf(\"error recreating Instance %s: %v\", i.ID, err)","handlingStrategy":"retry","validationCode":"// before calling rolling-update/delete\nmigExists, _ := gcloudInstanceGroupExists(project, zone, migName)\ninstanceExists, _ := gcloudInstanceExists(project, zone, instanceName)\nif !migExists || !instanceExists {\n    // skip recreation; instance/MIG already gone\n}","typeGuard":"func isNotFoundErr(err error) bool {\n    if gerr, ok := err.(*googleapi.Error); ok {\n        return gerr.Code == 404\n    }\n    return false\n}","tryCatchPattern":"result, err := recreateInstance(project, zone, migName, instanceID)\nvar gerr *googleapi.Error\nif errors.As(err, &gerr) {\n    switch {\n    case gerr.Code == 404:\n        log.Info(\"instance/MIG gone; treating as deleted\")\n    case gerr.Code == 429 || gerr.Code >= 500:\n        retryWithBackoff(recreateInstance)\n    default:\n        log.Errorf(\"recreate failed for %s: %v\", instanceID, err)\n    }\n}","preventionTips":["Do not delete MIG members manually in the GCP console while a rolling-update is running.","Ensure the kOps service account has roles/compute.instanceAdmin.v1.","Re-run rolling-update after transient failures instead of manual intervention.","Watch klog output for 'Instance not found, assuming deleted' to confirm not-found handling."],"tags":["gce","gcp","instance-group","rolling-update","kops"],"backgroundTag":"gce-mig-recreate-instance-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}