kubernetes/kops · error

error listing secrets for mirror: %v

Error message

error listing secrets for mirror: %v

What it means

VFSSecretStore.MirrorTo copies every secret from the store's VFS basedir to a target basedir. Before copying, it enumerates the source secrets with ListSecrets(); if that enumeration fails (the underlying ReadDir on the VFS path returns a non-NotExist error), the error is wrapped as 'error listing secrets for mirror'. This is a fail-fast guard: without a reliable listing, mirroring cannot proceed safely.

Source

Thrown at upup/pkg/fi/secrets/vfs_secretstore.go:59

	c := &VFSSecretStore{
		VFSSecretStoreReader: VFSSecretStoreReader{
			basedir: basedir,
		},
		cluster: cluster,
	}
	return c
}

func (c *VFSSecretStore) MirrorTo(ctx context.Context, basedir vfs.Path) error {
	if basedir.Path() == c.basedir.Path() {
		klog.V(2).Infof("Skipping mirror of secret store from %q to %q (same path)", c.basedir, basedir)
		return nil
	}
	klog.V(2).Infof("Mirroring secret store from %q to %q", c.basedir, basedir)

	secrets, err := c.ListSecrets()
	if err != nil {
		return fmt.Errorf("error listing secrets for mirror: %v", err)
	}

	for _, name := range secrets {
		secret, err := c.FindSecret(name)
		if err != nil {
			return fmt.Errorf("error reading secret %q for mirror: %v", name, err)
		}

		if secret == nil {
			return fmt.Errorf("unable to find secret %q for mirror", name)
		}

		p := BuildVfsSecretPath(basedir, name)

		acl, err := acls.GetACL(ctx, p, c.cluster)
		if err != nil {
			return fmt.Errorf("error building acl for secret %q for mirror: %v", name, err)
		}

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Verify credentials for the state store backend (e.g. `aws sts get-caller-identity`) and re-run
  2. Check the secrets basedir is reachable and readable: `kops get secrets` or an aws s3 ls on the prefix
  3. Inspect the wrapped %v cause; fix the underlying vfs/storage error (permissions, network, bucket existence)
  4. If migrating, mirror from a machine that has read access to the cluster state store

Example fix

// before: mirroring with no credentials
ctx := context.Background()
err := store.MirrorTo(ctx, targetDir) // error listing secrets for mirror: AccessDenied
// after: establish credentials first
ctx := context.Background()
if _, err := awsConfig.LoadDefaultConfig(ctx); err != nil { return err }
err := store.MirrorTo(ctx, targetDir)
Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-check reachability of the secrets dir
if _, err := store.ListSecrets(); err != nil {
    return fmt.Errorf("secrets store unreachable: %v", err)
}

Try / catch

if err := store.MirrorTo(ctx, target); err != nil {
    if strings.Contains(err.Error(), "error listing secrets for mirror") {
        // inspect cause, fix credentials/network, retry once
        return retryAfterCredentialRefresh(err)
    }
    return err
}

Prevention

When it happens

Trigger: Calling MirrorTo when the source VFS path (e.g. s3://cluster/pki/secrets, a GCS path, or a local dir) exists but ReadDir fails: no cloud credentials, network outage, permission denied on the bucket/prefix, or malformed VFS path. Notably os.IsNotExist is tolerated (returns empty list), so this error means a real read failure.

Common situations: Expired or missing AWS/GCP credentials (kops export/replace secrets against remote state), IAM policy revoking s3:ListBucket, DNS/proxy issues reaching object storage, or state store pointing at a deleted/renamed bucket.

Related errors

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/ac6aa24856ac31f1. Report an issue: GitHub.