{"record":{"id":"7ce19cf0a1412dbd","repo":"kubernetes/kops","slug":"unable-to-find-secret-q-for-mirror","errorCode":null,"errorMessage":"unable to find secret %q for mirror","messagePattern":"unable to find secret %q for mirror","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"upup/pkg/fi/secrets/vfs_secretstore.go","lineNumber":69,"sourceCode":"\tif basedir.Path() == c.basedir.Path() {\n\t\tklog.V(2).Infof(\"Skipping mirror of secret store from %q to %q (same path)\", c.basedir, basedir)\n\t\treturn nil\n\t}\n\tklog.V(2).Infof(\"Mirroring secret store from %q to %q\", c.basedir, basedir)\n\n\tsecrets, err := c.ListSecrets()\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error listing secrets for mirror: %v\", err)\n\t}\n\n\tfor _, name := range secrets {\n\t\tsecret, err := c.FindSecret(name)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error reading secret %q for mirror: %v\", name, err)\n\t\t}\n\n\t\tif secret == nil {\n\t\t\treturn fmt.Errorf(\"unable to find secret %q for mirror\", name)\n\t\t}\n\n\t\tp := BuildVfsSecretPath(basedir, name)\n\n\t\tacl, err := acls.GetACL(ctx, p, c.cluster)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error building acl for secret %q for mirror: %v\", name, err)\n\t\t}\n\n\t\tklog.Infof(\"mirroring secret %s -> %s\", name, p)\n\n\t\terr = createSecret(ctx, secret, p, acl, true)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error writing secret %q for mirror: %v\", name, err)\n\t\t}\n\t}\n\n\treturn nil","sourceCodeStart":51,"sourceCodeEnd":87,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/secrets/vfs_secretstore.go#L51-L87","documentation":"MirrorTo lists secret names via ListSecrets(), then immediately re-reads each with FindSecret(). FindSecret returns (nil, nil) when the file does not exist (IsNotExist maps to nil). If a listed secret disappears between listing and reading, MirrorTo treats it as an inconsistency and returns 'unable to find secret %q for mirror' rather than silently skipping it.","triggerScenarios":"A secret file is deleted (or the listing includes a directory/placeholder entry that is not a readable file) between ReadDir and the subsequent FindSecret read, or another concurrent kops process removed it.","commonSituations":"Two kops commands operating on the same state store concurrently (one pruning secrets while another mirrors), stale S3/GCS eventual-consistency edge cases, or a hand-cleaned secrets directory where a non-secret entry (e.g. directory) appeared in the listing.","solutions":["Re-run the mirror; the transient race usually resolves","Ensure only one kops process mutates the state store at a time","Inspect the secrets prefix for non-file entries or stale objects and remove them","If the secret was intentionally deleted, the listing is stale — retry after the deletion propagates"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Ensure stable listing: list twice and require identical sets before mirroring\nfirst, _ := store.ListSecrets()\nsecond, _ := store.ListSecrets()\nif !reflect.DeepEqual(first, second) { return errors.New(\"secrets store is changing; retry later\") }","typeGuard":null,"tryCatchPattern":"for attempt := 0; attempt < 3; attempt++ {\n    err := store.MirrorTo(ctx, target)\n    if err == nil || !strings.Contains(err.Error(), \"unable to find secret\") { return err }\n    time.Sleep(2 * time.Second)\n}\nreturn errors.New(\"mirror kept racing with concurrent mutation\")","preventionTips":["Serialize state-store mutations (lock or single operator)","Do not delete secrets while mirroring","Use FindSecret() semantics if you want skip-if-deleted behavior instead of hard failure","Clean non-file entries out of the secrets prefix"],"tags":["vfs","secrets","race-condition","mirror"],"backgroundTag":"secret-not-found","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}