{"record":{"id":"519e03d06f25550b","repo":"kubernetes/kops","slug":"error-fetching-gce-instance-w","errorCode":null,"errorMessage":"error fetching GCE instance: %w","messagePattern":"error fetching GCE instance: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/nodeidentity/gce/identify.go","lineNumber":217,"sourceCode":"\t\t\tcase kops.InstanceGroupRoleAPIServer:\n\t\t\t\tlabels[nodelabels.RoleLabelAPIServer16] = \"\"\n\t\t\tdefault:\n\t\t\t\tklog.Warningf(\"unknown node role %q for server %q\", role, instance.SelfLink)\n\t\t\t}\n\t\t}\n\t}\n\tif igName != \"\" {\n\t\tlabels[kops.NodeLabelInstanceGroup] = igName\n\t}\n\tinfo.Labels = labels\n\treturn info, nil\n}\n\n// getInstance queries GCE for the instance with the specified name, returning an error if not found\nfunc (i *nodeIdentifier) getInstance(zone string, instanceName string) (*compute.Instance, error) {\n\tinstance, err := i.computeService.Instances.Get(i.project, zone, instanceName).Do()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error fetching GCE instance: %w\", err)\n\t}\n\n\treturn instance, nil\n}\n\n// getInstanceTemplate queries GCE for the IG Template with the specified name, returning an error if not found\nfunc (i *nodeIdentifier) getInstanceTemplate(name string) (*compute.InstanceTemplate, error) {\n\tt, err := i.computeService.InstanceTemplates.Get(i.project, name).Do()\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"error fetching GCE instance group template %q: %v\", name, err)\n\t}\n\n\treturn t, nil\n}\n\n// getMIG queries GCE for the MIG with the specified name, returning an error if not found\nfunc (i *nodeIdentifier) getMIG(zone string, migName string) (*compute.InstanceGroupManager, error) {\n\tmig, err := i.computeService.InstanceGroupManagers.Get(i.project, zone, migName).Do()","sourceCodeStart":199,"sourceCodeEnd":235,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/pkg/nodeidentity/gce/identify.go#L199-L235","documentation":"getInstance wraps the GCE compute.Instances.Get call; any error from the API (not-found, permission denied, quota, network) is wrapped with this message. It means the identifier could not fetch the instance resource for the given project/zone/name, and the underlying GCE API error is preserved via %w for inspection.","triggerScenarios":"IdentifyNode calls i.getInstance(zone, instanceName) and the GCE API call fails: instance deleted between parse and fetch, wrong zone in providerID, service account lacking compute.instances.get, API disabled, or transient 5xx/network error.","commonSituations":"Node's VM was deleted (stale Node object) — googleapi 404; IAM service account missing roles/compute.viewer; Compute Engine API disabled in the project; firewall/proxy blocking googleapis.com; transient GCE API outages.","solutions":["Unwrap the error in logs: if 404, the instance is gone — delete the stale k8s Node object.","If 403/PERMISSION_DENIED, grant the controller's service account compute.instances.get (roles/compute.viewer) in the node project.","Ensure the Compute Engine API is enabled (gcloud services enable compute.googleapis.com).","For transient errors (5xx, timeouts, rate limits), add retry with exponential backoff around IdentifyNode/getInstance."],"exampleFix":"// before\ninstance, err := i.computeService.Instances.Get(i.project, zone, instanceName).Do()\nif err != nil { return nil, fmt.Errorf(\"error fetching GCE instance: %w\", err) }\n// after — retry transient failures\nvar instance *compute.Instance\nerr := wait.ExponentialBackoff(defaultBackoff, func() (bool, error) {\n    inst, err := i.computeService.Instances.Get(i.project, zone, instanceName).Do()\n    if err == nil { instance = inst; return true, nil }\n    if isNotFound(err) { return false, err } // permanent\n    return false, nil                         // transient: retry\n})\nif err != nil { return nil, fmt.Errorf(\"error fetching GCE instance: %w\", err) }","handlingStrategy":"try-catch","validationCode":"// pre-flight: verify IAM + API before identification loop\n_, err := computeSvc.Projects.Get(project).Do()\nif err != nil {\n    return fmt.Errorf(\"GCE API/IAM preflight failed for project %s: %w\", project, err)\n}","typeGuard":"func isNotFound(err error) bool {\n    var apiErr *googleapi.Error\n    return errors.As(err, &apiErr) && apiErr.Code == 404\n}","tryCatchPattern":"info, err := identifier.IdentifyNode(ctx, node)\nif err != nil {\n    var apiErr *googleapi.Error\n    if errors.As(err, &apiErr) && errors.Is(err, errWrapped) && apiErr.Code == 404 {\n        // instance gone: delete stale Node\n        return deleteNode(node)\n    }\n    if apiErr != nil && (apiErr.Code >= 500 || apiErr.Code == 429) {\n        return requeueWithBackoff(err) // transient: retry\n    }\n    return err\n}","preventionTips":["Check unwrapped googleapi.Error codes: 404 = stale node, 403 = IAM, 5xx/429 = transient retry.","Grant compute.instances.get (roles/compute.viewer) to the controller service account.","Enable the Compute Engine API in the project.","Add exponential backoff for transient GCE API failures; monitor googleapis.com reachability."],"tags":["gce","api-error","node-identity","iam","retry"],"backgroundTag":"gce-api-error","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}