kubernetes/kops · error

failed to update pool membership: %v

Error message

failed to update pool membership: %v

What it means

Wrapped failure from v2pools.UpdateMember inside a memberBackoff retry loop in updateMemberInPool. The Octavia API rejected the member update with a status other than the specially-handled 409 Conflict (immutable/PENDING state). The error contains the underlying gophercloud failure.

Source

Thrown at upup/pkg/fi/cloudup/openstack/loadbalancer.go:387

func updateMemberInPool(c OpenstackCloud, poolID string, memberID string, opts v2pools.UpdateMemberOptsBuilder) (association *v2pools.Member, err error) {
	if c.LoadBalancerClient() == nil {
		return nil, fmt.Errorf("loadbalancer support not available in this deployment")
	}

	done, err := vfs.RetryWithBackoff(memberBackoff, func() (bool, error) {
		association, err = v2pools.UpdateMember(context.TODO(), c.LoadBalancerClient(), poolID, memberID, opts).Extract()
		if err != nil {
			// member not found anymore
			if isNotFound(err) {
				return true, nil
			}
			// pool is currently in immutable state, try to retry
			if gophercloud.ResponseCodeIs(err, http.StatusConflict) {
				klog.Infof("got error %v retrying...", http.StatusConflict)
				return false, nil
			}
			return false, fmt.Errorf("failed to update pool membership: %v", err)
		}
		return true, nil
	})
	if !done {
		if err == nil {
			err = wait.ErrWaitTimeout
		}
		return association, err
	}
	return association, nil
}

func (c *openstackCloud) AssociateToPool(server *servers.Server, poolID string, opts v2pools.CreateMemberOpts) (association *v2pools.Member, err error) {
	return associateToPool(c, server, poolID, opts)
}

func associateToPool(c OpenstackCloud, server *servers.Server, poolID string, opts v2pools.CreateMemberOpts) (association *v2pools.Member, err error) {
	if c.LoadBalancerClient() == nil {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the wrapped %v error: 404 means the member is gone — remove it from the desired state rather than retrying.
2. If timeouts follow repeated 409 (handled earlier in the code), reduce update concurrency or wait for the LB to leave PENDING_UPDATE before re-running.
3. Verify the credentials can update members (load-balancer_member role) and the pool ID is correct.
4. Increase memberBackoff duration for large pools where Octavia updates are slow.

pool, err := cloud.GetPool(poolID)
if err != nil {
	return fmt.Errorf("pool %s not accessible: %w", poolID, err)
}
if pool.ProvisioningStatus != "ACTIVE" {
	return fmt.Errorf("pool %s is %s; defer member update", poolID, pool.ProvisioningStatus)
}

err := cloud.UpdateMemberInPool(poolID, memberID, opts)
if err != nil {
	if strings.Contains(err.Error(), "404") || strings.Contains(err.Error(), "Not Found") {
		return reconcileMemberRemoved(memberID) // treat as deleted
	}
	if strings.Contains(err.Error(), "failed to update pool membership") {
		return retryAfterPendingClears(poolID, opts)
	}
	return err
}

When it happens

Trigger: Calling UpdateMemberInPool while the pool or load balancer is in PENDING_UPDATE/PENDING_CREATE state (API returns 409 repeatedly until timeout), on 404 when the member vanished, or on 4xx/5xx auth/permission errors — any non-conflict failure escapes as this message.

Common situations: Rolling node updates racing with Octavia's own reconciles; frequent member churn (autoscaling) keeping the pool constantly in immutable state until retries exhaust; stale member IDs after node replacement.

Related errors

• Failed to create member: %v
• Failed to update member: %v
• failed to build load balancer client: %w
• cluster configured to use octavia, but router was not config
• error building lb client: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/0accdbd5d2f7aabb. Report an issue: GitHub.