kubernetes/kops · error

error querying bucket %q: %v

Error message

error querying bucket %q: %v

What it means

GetACL queries a GCS bucket's metadata to determine whether it enforces uniform bucket-level access. When the storage.Client call to Bucket(bucketName).Attrs(ctx) fails (network error, missing bucket, insufficient permissions), the underlying GCS error is wrapped as `error querying bucket %q: %v` and returned so callers know which bucket could not be inspected.

Source

Thrown at util/pkg/vfs/acls/gce/storage.go:56

func (s *gcsAclStrategy) GetACL(ctx context.Context, p vfs.Path, cluster *kops.Cluster) (vfs.ACL, error) {
	if cluster.GetCloudProvider() != kops.CloudProviderGCE {
		return nil, nil
	}
	gcsPath, ok := p.(*vfs.GSPath)
	if !ok {
		return nil, nil
	}

	bucketName := gcsPath.Bucket()
	client, err := gcsPath.Client(ctx)
	if err != nil {
		return nil, err
	}

	// TODO: Cache?
	bucket, err := client.Bucket(bucketName).Attrs(ctx)
	if err != nil {
		return nil, fmt.Errorf("error querying bucket %q: %v", bucketName, err)
	}

	if bucket.UniformBucketLevelAccess.Enabled {
		klog.V(8).Infof("bucket gs://%s has bucket-policy only; won't try to set ACLs", bucketName)
		return nil, nil
	}

	// TODO: Cache?
	cloud, err := cloudup.BuildCloud(cluster)
	if err != nil {
		return nil, err
	}

	serviceAccount, err := cloud.(gce.GCECloud).ServiceAccount()
	if err != nil {
		return nil, err
	}

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Verify the bucket name in your state store URL exactly matches an existing bucket (gsutil ls or gcloud storage buckets describe).
  2. Grant the service account storage.buckets.get access (roles/storage.admin or at minimum roles/storage.objectViewer plus bucketViewer/legacyBucketReader).
  3. Check GOOGLE_APPLICATION_CREDENTIALS points to a valid, non-expired service-account key.
  4. Retry if the underlying error is transient (timeouts, 5xx, 429 rate limits).

Example fix

// before (insufficient perms)
client, _ := storage.NewClient(ctx)
acl, err := vfs.GetACL(ctx, path, cluster) // error querying bucket "my-bucket": googleapi: Error 403
// after: grant IAM first
gcloud storage buckets add-iam-policy-binding gs://my-bucket \
  --member=serviceAccount:sa@project.iam.gserviceaccount.com \
  --role=roles/storage.admin
Defensive patterns

Strategy: try-catch

Validate before calling

// Go: check bucket exists & is accessible before GetACL
client, err := storage.NewClient(ctx)
if err != nil { return err }
_, err = client.Bucket("my-bucket").Attrs(ctx)
if err != nil {
    if e, ok := err.(*googleapi.Error); ok && e.Code == 404 {
        return fmt.Errorf("bucket my-bucket does not exist")
    }
    return err
}

Type guard

func isGoogleAPIError(err error) (*googleapi.Error, bool) {
    var ge *googleapi.Error
    if errors.As(err, &ge) { return ge, true }
    return nil, false
}

Try / catch

acl, err := vfs.GetACL(ctx, path, cluster)
if err != nil {
    var ge *googleapi.Error
    if errors.As(err, &ge) {
        switch ge.Code {
        case 404: // bucket missing: create it or fix state store URL
        case 403: // grant storage.buckets.get IAM
        case 429, 500, 503: // retry with backoff
        }
    }
    return err
}

Prevention

When it happens

Trigger: Calling vfs GetACL against a gs:// path whose bucket name is wrong/typo'd, the bucket does not exist or was deleted, the credentials lack storage.buckets.get (e.g. missing storage.admin or objectViewer+bucketViewer roles), or a transient network/API failure occurs while calling the GCS Storage API.

Common situations: Misconfigured KOPS_STATE_STORE gs:// bucket name; using service-account credentials that only have object-level permissions on a bucket with uniform bucket-level access; buckets removed between cluster operations; flaky connectivity or GCS API outages during `kops get/export` against a GCS state store.

Related errors

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/d4348815848585dd. Report an issue: GitHub.