kubernetes/kops · error

failed to list loadbalancers: %s

Error message

failed to list loadbalancers: %s

What it means

Inside ListLB, loadbalancers.List(...).AllPages is executed under vfs.RetryWithBackoff; any pagination/API failure is wrapped as 'failed to list loadbalancers: %s' (extraction failures get a separate message). It means the Octavia list endpoint failed — auth, endpoint, or server error — after retries, not that zero LBs were found.

Source

Thrown at upup/pkg/fi/cloudup/openstack/loadbalancer.go:274

	}
	return lb, nil
}

// ListLBs will list load balancers
func (c *openstackCloud) ListLBs(opt loadbalancers.ListOptsBuilder) (lbs []loadbalancers.LoadBalancer, err error) {
	return listLBs(c, opt)
}

func listLBs(c OpenstackCloud, opt loadbalancers.ListOptsBuilder) (lbs []loadbalancers.LoadBalancer, err error) {
	if c.LoadBalancerClient() == nil {
		// skip error because cluster delete will otherwise fail
		return lbs, nil
	}

	done, err := vfs.RetryWithBackoff(readBackoff, func() (bool, error) {
		allPages, err := loadbalancers.List(c.LoadBalancerClient(), opt).AllPages(context.TODO())
		if err != nil {
			return false, fmt.Errorf("failed to list loadbalancers: %s", err)
		}
		lbs, err = loadbalancers.ExtractLoadBalancers(allPages)
		if err != nil {
			return false, fmt.Errorf("failed to extract loadbalancer pages: %s", err)
		}
		return true, nil
	})
	if !done {
		if err == nil {
			err = wait.ErrWaitTimeout
		}
		return lbs, err
	}
	return lbs, nil
}

func (c *openstackCloud) GetLBStats(loadbalancerID string) (stats *loadbalancers.Stats, err error) {
	return getLBStats(c, loadbalancerID)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-authenticate and confirm the token is scoped to the correct project (openstack token issue)
2. Check the load-balancer endpoint exists and is reachable (openstack endpoint list / curl the octavia URL)
3. Inspect the wrapped error: 403 → fix RBAC policy; 429 → slow down/increase rate limits
4. Narrow the ListOpts filters to reduce page size if pagination times out on large projects
5. Retry during Octavia recovery — readBackoff already retries transient failures

// ensure a fresh, correctly-scoped token before listing
if tokenExpired(client.ProviderClient) {
	if err := reauthenticate(client.ProviderClient, authOpts); err != nil {
		return fmt.Errorf("re-authentication failed before listing loadbalancers: %v", err)
	}
}

if _, err := cloud.ListLB(loadbalancers.ListOpts{ProjectID: projectID}); err != nil {
	if strings.Contains(err.Error(), "401") || strings.Contains(err.Error(), "Unauthorized") {
		// token scoping/expiry issue: re-auth and retry once
		reauthenticate(client.ProviderClient, authOpts)
	} else if strings.Contains(err.Error(), "429") {
		// rate limited: back off longer before next attempt
		time.Sleep(rateLimitBackoff)
	} else {
		log.Warningf("failed to list loadbalancers: %s", err)
	}
}

When it happens

Trigger: Listing all loadbalancers in a project (e.g., during reconciliation or garbage collection) when Octavia returns 401/403 (bad token/scoped project), 404 (endpoint missing), pagination failure on very large projects, or transient 429/5xx responses.

Common situations: Keystone token expiry or wrong project scope for the LB client; misconfigured load-balancer endpoint in the catalog; projects with thousands of LBs hitting pagination timeouts; Octavia rate limiting under load.

Related errors

• error deleting loadbalancer: %v
• error creating loadbalancer: %v
• failed to list members: %v
• failed to list pools: %v
• failed to list listeners: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/03cb9491a490ad45. Report an issue: GitHub.