kubernetes/kops · error

error describing volumes: %v

Error message

error describing volumes: %v

What it means

findEtcdStatus queries the OpenStack Cinder v3 API for volumes tagged with etcd cluster metadata, filtering by the cluster's cloud tags. When c.ListVolumes(opt) fails (API call, auth, or pagination error), the underlying error is wrapped as "error describing volumes: %v" and returned to findClusterStatus. This aborts cluster status discovery since etcd state cannot be determined.

Source

Thrown at upup/pkg/fi/cloudup/openstack/status.go:69

		return nil, err
	}
	status := &kops.ClusterStatus{
		EtcdClusters: etcdStatus,
	}
	klog.V(2).Infof("Cluster status (from cloud): %v", fi.DebugAsJsonString(status))
	return status, nil
}

// findEtcdStatus discovers the status of etcd, by looking for the tagged etcd volumes
func findEtcdStatus(c OpenstackCloud, cluster *kops.Cluster) ([]kops.EtcdClusterStatus, error) {
	statusMap := make(map[string]*kops.EtcdClusterStatus)
	klog.V(2).Infof("Querying Openstack for etcd volumes")
	opt := cinderv3.ListOpts{
		Metadata: c.GetCloudTags(),
	}
	volumes, err := c.ListVolumes(opt)
	if err != nil {
		return nil, fmt.Errorf("error describing volumes: %v", err)
	}

	for _, volume := range volumes {
		volumeID := volume.ID

		etcdClusterName := ""
		var etcdClusterSpec *etcd.EtcdClusterSpec

		master := false
		for k, v := range volume.Metadata {
			if strings.HasPrefix(k, TagNameEtcdClusterPrefix) {
				etcdClusterName := strings.TrimPrefix(k, TagNameEtcdClusterPrefix)
				etcdClusterSpec, err = etcd.ParseEtcdClusterSpec(etcdClusterName, v)
				if err != nil {
					return nil, fmt.Errorf("error parsing etcd cluster tag %q on volume %q: %v", v, volumeID, err)
				}
			} else if k == TagNameRolePrefix+TagRoleControlPlane || k == TagNameRolePrefix+TagRoleMaster {
				master = true

View on GitHub (pinned to 4c8573c808)

Solutions

1. Verify OpenStack credentials and that `openstack volume list` works with the same env/auth
2. Check the wrapped underlying error message for 401 (re-authenticate) vs timeout (network/service)
3. Retry the kops command once the cinder service is healthy
4. Confirm the cluster's cloud tags in the spec are valid metadata keys

Example fix

// before
volumes, err := c.ListVolumes(opt)
if err != nil {
	return nil, fmt.Errorf("error describing volumes: %v", err)
}
// after (preserve %w for wrapping and add context)
volumes, err := c.ListVolumes(opt)
if err != nil {
	return nil, fmt.Errorf("error describing volumes: %w", err)
}

// Go: check credentials/API reachability before running kops
if err := exec.Command("openstack", "volume", "list", "--limit", "1").Run(); err != nil {
	log.Fatal("OpenStack cinder API unreachable or credentials invalid")
}

func isAuthFailure(err error) bool {
	return err != nil && strings.Contains(err.Error(), "401")
}

status, err := findClusterStatus(cloud)
if err != nil {
	var wrapped *fmt.WrapError
	if strings.Contains(err.Error(), "error describing volumes") && isAuthFailure(err) {
		// refresh token and retry
	}
	return fmt.Errorf("cluster status unavailable: %w", err)
}

When it happens

Trigger: OpenStack API endpoint unreachable, invalid/expired Keystone token, neutron/cinder service outage, or a malformed ListOpts metadata filter causing the cinder API to reject the request.

Common situations: Expired cloud credentials in the environment, misconfigured OS_* environment variables, network partition between the kops host and the OpenStack cloud, or cinder service down during cluster upgrade/status checks.

Related errors

• failed to get all volumes by region from %s: %v
• volume tag split failed, too few components for tag %q on vo
• error parsing etcd cluster tag %q on volume %q: %v
• could not determine etcd cluster type for volume: %s
• failed to extract storage availability zones: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/83825e29bddd5faf. Report an issue: GitHub.