kubernetes/kops · error

updating back-end server IPs for load-balancer %s: %w

Error message

updating back-end server IPs for load-balancer %s: %w

What it means

kOps wraps any failure from the Scaleway SDK `lbService.SetBackendServers` call (which updates which server IPs a load-balancer back-end forwards traffic to) with the load-balancer's name. The wrapped error comes from the Scaleway LB API over the network. It is thrown while reconciling the LoadBalancerBackend task in RenderScw when the back-end already exists but its server IP list must change.

Source

Thrown at upup/pkg/fi/cloudup/scalewaytasks/lb_backend.go:163

			BackendID:            fi.ValueOf(actual.ID),
			Name:                 fi.ValueOf(actual.Name),
			ForwardProtocol:      lb.Protocol(fi.ValueOf(expected.ForwardProtocol)),
			ForwardPort:          fi.ValueOf(expected.ForwardPort),
			ForwardPortAlgorithm: lb.ForwardPortAlgorithm(fi.ValueOf(expected.ForwardPortAlgorithm)),
			StickySessions:       lb.StickySessionsType(fi.ValueOf(expected.StickySessions)),
			ProxyProtocol:        lb.ProxyProtocol(fi.ValueOf(expected.ProxyProtocol)),
		})
		if err != nil {
			return fmt.Errorf("updating back-end for load-balancer %s: %w", fi.ValueOf(actual.LoadBalancer.Name), err)
		}

		_, err = lbService.SetBackendServers(&lb.ZonedAPISetBackendServersRequest{
			Zone:      zone,
			BackendID: fi.ValueOf(actual.ID),
			ServerIP:  controlPlanesIPs,
		})
		if err != nil {
			return fmt.Errorf("updating back-end server IPs for load-balancer %s: %w", fi.ValueOf(actual.LoadBalancer.Name), err)
		}

	} else {

		backendCreated, err := lbService.CreateBackend(&lb.ZonedAPICreateBackendRequest{
			Zone:                 zone,
			LBID:                 fi.ValueOf(expected.LoadBalancer.LBID),
			Name:                 fi.ValueOf(expected.Name),
			ForwardProtocol:      lb.Protocol(fi.ValueOf(expected.ForwardProtocol)),
			ForwardPort:          fi.ValueOf(expected.ForwardPort),
			ForwardPortAlgorithm: lb.ForwardPortAlgorithm(fi.ValueOf(expected.ForwardPortAlgorithm)),
			StickySessions:       lb.StickySessionsType(fi.ValueOf(expected.StickySessions)),
			HealthCheck: &lb.HealthCheck{
				CheckMaxRetries: 5,
				TCPConfig:       &lb.HealthCheckTCPConfig{},
				Port:            fi.ValueOf(expected.ForwardPort),
				CheckTimeout:    scw.TimeDurationPtr(3000),
				CheckDelay:      scw.TimeDurationPtr(1001),

View on GitHub (pinned to 4c8573c808)

Solutions

1. Rerun `kops update cluster` — Scaleway LB errors are often transient; the retry is idempotent
2. Verify the load-balancer and back-end still exist in the Scaleway console for the cluster's zone
3. Check SCALEWAY API credentials/permissions (lb-editor role needed) and network reachability
4. Check Scaleway status page for LB API incidents
5. Increase retry/timeout or file a bug if a specific API error code (e.g. quota exceeded) recurs

Example fix

// before (no handling of transient failure)
_, err = lbService.SetBackendServers(...)
// after (retry transient failures)
var resp *lb.Backend
var rerr error
for i := 0; i < 3; i++ {
    resp, rerr = lbService.SetBackendServers(req)
    if rerr == nil || !scw.IsTransientError(rerr) { break }
    time.Sleep(time.Duration(i+1) * 5 * time.Second)
}

// before update: ensure backend and LB exist and are ready
backend, _, err := lbService.GetBackend(&lb.ZonedAPIGetBackendRequest{Zone: zone, BackendID: backendID})
if err != nil { return fmt.Errorf("backend %s missing: %w", backendID, err) }
if _, err := lbService.WaitForLb(&lb.ZonedAPIWaitForLBRequest{Zone: zone, LBID: lbID}); err != nil { return err }

var serr *scw.ResponseError
if errors.As(err, &serr) {
    switch serr.Status {
    case 404:
        // recreate backend instead of updating
    case 429, 500, 502, 503, 504:
        // transient: retry with backoff
    default:
        return err // permanent: surface to user
    }
}

When it happens

Trigger: RenderScw finds an existing back-end (`actual`) and calls SetBackendServers with the new control-plane private IPs; the Scaleway Zoned API rejects or times out the request — e.g. invalid backend ID, LB in a transient state, quota on back-end servers, or network/auth failure.

Common situations: Control-plane instance set was resized or replaced so back-end IPs must be updated; Scaleway API transient 5xx/timeout during cluster reconcile; LB was deleted out-of-band; Scaleway credentials expired; scw zone mismatch.

Related errors

• waiting for load-balancer: %w
• deleting load-balancer %s: %w
• creating back-end for load-balancer %s: %w
• getting cluster servers for load-balancer's back-end: %w
• getting IP of server %s for load-balancer's back-end: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/f0579fa8840d7a8f. Report an issue: GitHub.