kubernetes/kops · error

unable to find secret %q for mirror

Error message

unable to find secret %q for mirror

What it means

MirrorTo lists secret names via ListSecrets(), then immediately re-reads each with FindSecret(). FindSecret returns (nil, nil) when the file does not exist (IsNotExist maps to nil). If a listed secret disappears between listing and reading, MirrorTo treats it as an inconsistency and returns 'unable to find secret %q for mirror' rather than silently skipping it.

Source

Thrown at upup/pkg/fi/secrets/vfs_secretstore.go:69

	if basedir.Path() == c.basedir.Path() {
		klog.V(2).Infof("Skipping mirror of secret store from %q to %q (same path)", c.basedir, basedir)
		return nil
	}
	klog.V(2).Infof("Mirroring secret store from %q to %q", c.basedir, basedir)

	secrets, err := c.ListSecrets()
	if err != nil {
		return fmt.Errorf("error listing secrets for mirror: %v", err)
	}

	for _, name := range secrets {
		secret, err := c.FindSecret(name)
		if err != nil {
			return fmt.Errorf("error reading secret %q for mirror: %v", name, err)
		}

		if secret == nil {
			return fmt.Errorf("unable to find secret %q for mirror", name)
		}

		p := BuildVfsSecretPath(basedir, name)

		acl, err := acls.GetACL(ctx, p, c.cluster)
		if err != nil {
			return fmt.Errorf("error building acl for secret %q for mirror: %v", name, err)
		}

		klog.Infof("mirroring secret %s -> %s", name, p)

		err = createSecret(ctx, secret, p, acl, true)
		if err != nil {
			return fmt.Errorf("error writing secret %q for mirror: %v", name, err)
		}
	}

	return nil

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-run the mirror; the transient race usually resolves
2. Ensure only one kops process mutates the state store at a time
3. Inspect the secrets prefix for non-file entries or stale objects and remove them
4. If the secret was intentionally deleted, the listing is stale — retry after the deletion propagates

// Ensure stable listing: list twice and require identical sets before mirroring
first, _ := store.ListSecrets()
second, _ := store.ListSecrets()
if !reflect.DeepEqual(first, second) { return errors.New("secrets store is changing; retry later") }

for attempt := 0; attempt < 3; attempt++ {
    err := store.MirrorTo(ctx, target)
    if err == nil || !strings.Contains(err.Error(), "unable to find secret") { return err }
    time.Sleep(2 * time.Second)
}
return errors.New("mirror kept racing with concurrent mutation")

When it happens

Trigger: A secret file is deleted (or the listing includes a directory/placeholder entry that is not a readable file) between ReadDir and the subsequent FindSecret read, or another concurrent kops process removed it.

Common situations: Two kops commands operating on the same state store concurrently (one pruning secrets while another mirrors), stale S3/GCS eventual-consistency edge cases, or a hand-cleaned secrets directory where a non-secret entry (e.g. directory) appeared in the listing.

Related errors

• error listing secrets for mirror: %v
• error reading secret %q for mirror: %v
• error building acl for secret %q for mirror: %v
• error writing secret %q for mirror: %v
• error loading secret %q: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/7ce19cf0a1412dbd. Report an issue: GitHub.