kubernetes/kops · error

error updating Healthcheck %q: %v

Error message

error updating Healthcheck %q: %v

What it means

Insert only applies Port/RequestPath on create, so for an existing HTTPHealthcheck with changes to those fields the task issues a separate HTTPHealthChecks().Update. If Update errors it is wrapped with this message; a subsequent WaitForOp failure is wrapped with the same prefix.

Source

Thrown at upup/pkg/fi/cloudup/gcetasks/httphealthcheck.go:111

		if err != nil {
			return fmt.Errorf("error creating Healthcheck %q: %v", o.Name, err)
		}
		if err := t.Cloud.WaitForOp(r); err != nil {
			return fmt.Errorf("error creating Healthcheck: %v", err)
		}
		h.SelfLink = r.TargetLink
	} else if changes.Port != nil || changes.RequestPath != nil {
		// Insert only applies these on create, so reconcile changes to an existing check with a separate Update.
		o := &compute.HttpHealthCheck{
			Name:        fi.ValueOf(e.Name),
			Port:        fi.ValueOf(e.Port),
			RequestPath: fi.ValueOf(e.RequestPath),
		}

		klog.V(4).Infof("Updating Healthcheck %q", o.Name)
		r, err := t.Cloud.Compute().HTTPHealthChecks().Update(t.Cloud.Project(), o.Name, o)
		if err != nil {
			return fmt.Errorf("error updating Healthcheck %q: %v", o.Name, err)
		}
		if err := t.Cloud.WaitForOp(r); err != nil {
			return fmt.Errorf("error updating Healthcheck: %v", err)
		}
	}
	return nil
}

type terraformHTTPHealthcheck struct {
	Name        string  `cty:"name"`
	Port        *int64  `cty:"port"`
	RequestPath *string `cty:"request_path"`
}

func (_ *HTTPHealthcheck) RenderTerraform(t *terraform.TerraformTarget, a, e, changes *HTTPHealthcheck) error {
	tf := &terraformHTTPHealthcheck{
		Name:        *e.Name,
		Port:        e.Port,

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Fix the underlying wrapped error: 403 => grant compute.healthChecks.update; 409 => remove the conflicting concurrent operation and retry
  2. Validate the new Port/RequestPath values (port range, path begins with /) in the cluster spec
  3. Retry the apply after backoff for 429/5xx
  4. As a last resort, delete and recreate the health check via a fresh apply

Example fix

// before (invalid request path in cluster spec)
requestPath: "status"
// after
requestPath: "/healthz"
Defensive patterns

Strategy: try-catch

Validate before calling

// validate changed fields before update
if changes.Port != nil && (*changes.Port < 1 || *changes.Port > 65535) { return errors.New("invalid port") }
if changes.RequestPath != nil && !strings.HasPrefix(*changes.RequestPath, "/") { return errors.New("requestPath must start with /") }

Try / catch

var gerr *googleapi.Error
if errors.As(err, &gerr) {
	switch gerr.Code {
	case 409: // concurrent modification; retry after the other op finishes
	case 403: // add compute.healthChecks.update IAM
	default: // inspect payload for field validation errors
	}
}

Prevention

When it happens

Trigger: HTTPHealthChecks().Update(project, name, o) fails: IAM denial on compute.healthChecks.update, concurrent modification conflict, invalid new Port/RequestPath values, rate limiting, or API error.

Common situations: Changing requestPath/port in the kops cluster spec for an existing instance group health check; another process modified the health check concurrently; missing compute.healthChecks.update IAM permission.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/84b4a741193441d5. Report an issue: GitHub.