kubernetes/kops · error

failed to get all volumes by region from %s: %v

Error message

failed to get all volumes by region from %s: %v

What it means

findEtcdStatus discovers etcd cluster health by listing all block-storage volumes in the region (GetAllVolumesByRegion) and reading their kOps tags. If that listing fails, the error is wrapped with the region for context. Without the volume list, etcd cluster status cannot be determined and FindClusterStatus fails.

Source

Thrown at upup/pkg/fi/cloudup/do/cloud.go:327

// FindClusterStatus discovers the status of the cluster, by looking for the tagged etcd volumes
func (c *doCloudImplementation) FindClusterStatus(cluster *kops.Cluster) (*kops.ClusterStatus, error) {
	etcdStatus, err := findEtcdStatus(c, cluster)
	if err != nil {
		return nil, err
	}
	status := &kops.ClusterStatus{
		EtcdClusters: etcdStatus,
	}
	klog.V(2).Infof("Cluster status (from cloud): %v", fi.DebugAsJsonString(status))
	return status, nil
}

// findEtcdStatus discovers the status of etcd, by looking for the tagged etcd volumes
func findEtcdStatus(c *doCloudImplementation, cluster *kops.Cluster) ([]kops.EtcdClusterStatus, error) {
	statusMap := make(map[string]*kops.EtcdClusterStatus)
	volumes, err := c.GetAllVolumesByRegion()
	if err != nil {
		return nil, fmt.Errorf("failed to get all volumes by region from %s: %v", c.Region(), err)
	}

	for _, volume := range volumes {
		volumeID := volume.ID

		etcdClusterName := ""
		var etcdClusterSpec *etcd.EtcdClusterSpec

		for _, myTag := range volume.Tags {
			klog.V(8).Infof("findEtcdStatus status (from cloud): checking if volume with tag %q belongs to cluster", myTag)
			// check if volume belongs to this cluster.
			// tag will be in the format "KubernetesCluster:dev5-k8s-local" (where clusterName is dev5.k8s.local)
			clusterName := strings.ReplaceAll(cluster.Name, ".", "-")
			if strings.Contains(myTag, fmt.Sprintf("%s:%s", TagKubernetesClusterNamePrefix, clusterName)) {
				klog.V(10).Infof("findEtcdStatus cluster comparison matched for tag: %v", myTag)
				// this volume belongs to our cluster, add this to our etcdClusterSpec.
				// loop through the tags again and
				for _, volumeTag := range volume.Tags {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the wrapped error for 401/403 (fix token scopes) vs 429 (back off and retry).
2. Verify the cluster's DO region is correct and volumes exist there (`doctl compute volume list --region <slug>`).
3. Confirm network egress to the DO API from the kOps host.
4. Re-run `kops get cluster --full` / status command after transient failures — the read path is safe to retry.

_, _, err := client.Storage.ListVolumes(context.TODO(), region, nil)
if err != nil { return fmt.Errorf("cannot list volumes in %s: %w", region, err) }

status, err := cloud.FindClusterStatus(cluster)
if err != nil {
    if isRateLimited(err) { backoff(); retry() }
    return fmt.Errorf("etcd status unavailable: %w", err)
}

When it happens

Trigger: GetAllVolumesByRegion() (godo Storage.List, region-paginated) returns an error: API auth failure, rate limiting, network error, or an invalid/unsupported region slug.

Common situations: Wrong region configured for the cluster; DO API token lacking read scope on volumes; transient API outages; rate limits when many volumes/pages must be listed.

Related errors

• volume tag split failed, too few components for tag %q on vo
• error parsing etcd cluster tag %q on volume %q: %v
• could not determine etcd cluster type for volume: %s
• error storing InstanceGroup: %v
• failed to retrieve droplet %d: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/ae80999ef63884e1. Report an issue: GitHub.