kubernetes/kops · error

error listing server groups: %v

Error message

error listing server groups: %v

What it means

listServerGroups wraps the Gophercloud servergroups.List(...).AllPages call. When paging the Nova server-groups index fails (the initial request or any subsequent page), the error is wrapped as "error listing server groups: %v". kOps uses this to find existing server groups during cluster operations, so the operation aborts.

Source

Thrown at upup/pkg/fi/cloudup/openstack/server_group.go:69

		return i, err
	} else if done {
		return i, nil
	} else {
		return i, wait.ErrWaitTimeout
	}
}

func (c *openstackCloud) ListServerGroups(opts servergroups.ListOptsBuilder) ([]servergroups.ServerGroup, error) {
	return listServerGroups(c, opts)
}

func listServerGroups(c OpenstackCloud, opts servergroups.ListOptsBuilder) ([]servergroups.ServerGroup, error) {
	var sgs []servergroups.ServerGroup

	done, err := vfs.RetryWithBackoff(readBackoff, func() (bool, error) {
		allPages, err := servergroups.List(c.ComputeClient(), opts).AllPages(context.TODO())
		if err != nil {
			return false, fmt.Errorf("error listing server groups: %v", err)
		}

		r, err := servergroups.ExtractServerGroups(allPages)
		if err != nil {
			return false, fmt.Errorf("error extracting server groups from pages: %v", err)
		}
		sgs = r
		return true, nil
	})
	if err != nil {
		return sgs, err
	} else if done {
		return sgs, nil
	} else {
		return sgs, wait.ErrWaitTimeout
	}
}

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Run 'openstack server group list' with the same credentials/region to reproduce and see the raw HTTP error.
  2. Fix auth: refresh OS_* credentials/token; if 403, grant the project the required compute roles.
  3. Correct the identity/region/interface settings in the kOps OpenStack cloud config so the compute endpoint resolves.
  4. If rate-limited, reduce concurrent API calls or increase nova-api rate limits, then retry.
  5. Retry after transient outages; persistent errors indicate credential/endpoint misconfiguration.

Example fix

// before: endpoint missing due to wrong interface
authOpts.IdentityEndpoint = "https://keystone.example.com/v3"
// after: let gophercloud discover the compute endpoint in the right region
authOpts.Scope = &tokens.Scope{ProjectName: project, DomainName: domain}
provider, _ := openstack.NewClient(authOpts.IdentityEndpoint)
openstack.Authenticate(provider, authOpts)
computeClient, _ := openstack.NewComputeV2(provider, endpoints.EndpointOpts{Region: correctRegion, Availability: endpoints.AvailabilityPublic})
Defensive patterns

Strategy: retry

Validate before calling

// Pre-flight: can we reach the compute endpoint at all?
client, err := openstack.NewComputeV2(provider, endpoints.EndpointOpts{Region: region})
if err != nil {
	return fmt.Errorf("compute endpoint for region %q not available: %w", region, err)
}

Try / catch

sgs, err := cloud.ListServerGroups(opts)
if err != nil {
	if errors.Is(err, wait.ErrWaitTimeout) {
		// retries exhausted; fall back to treating as empty or fail loudly
		return fmt.Errorf("listing server groups timed out: %w", err)
	}
	var gerr gophercloud.ErrUnexpectedResponseCode
	if errors.As(err, &gerr) && (gerr.Actual == 401 || gerr.Actual == 403) {
		// re-authenticate once, then retry the listing
		if rerr := reauthenticate(provider); rerr == nil {
			sgs, err = cloud.ListServerGroups(opts)
		}
	}
	if err != nil {
		return err
	}
}

Prevention

When it happens

Trigger: servergroups.List(c.ComputeClient(), opts).AllPages(context.TODO()) errors: 401/403 auth failure against the compute endpoint, 404 missing compute service/endpoint, malformed ListOpts filter, API rate limiting (413/429), or a network timeout mid-pagination.

Common situations: Expired Keystone token during a long-running kOps command; misconfigured region or endpoint interface (public vs internal); nova-api briefly unavailable; cloud policy revoking compute:list_server_groups permission; OpenStack upgrade changing the server-groups API response shape so pagination breaks.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/149c3bedec4091e7. Report an issue: GitHub.