kubernetes/kops · error

unable to write secret: %v

Error message

unable to write secret: %v

What it means

ReplaceSecret overwrites an existing secret by calling createSecret with replace=true, which uses p.WriteFile on the VFS path. If that write fails — storage error, permission denied, ACL rejection — the error is wrapped as 'unable to write secret'. The replacement did not take effect.

Source

Thrown at upup/pkg/fi/secrets/vfs_secretstore.go:168

		klog.Fatalf("unable to load secret immediately after creation %v: %v", p, err)
		return nil, false, err
	}
	return s, true, nil
}

func (c *VFSSecretStore) ReplaceSecret(id string, secret *fi.Secret) (*fi.Secret, error) {
	ctx := context.TODO()

	p := c.buildSecretPath(id)

	acl, err := acls.GetACL(ctx, p, c.cluster)
	if err != nil {
		return nil, err
	}

	err = createSecret(ctx, secret, p, acl, true)
	if err != nil {
		return nil, fmt.Errorf("unable to write secret: %v", err)
	}

	// Confirm the secret exists
	s, err := c.loadSecret(ctx, p)
	if err != nil {
		return nil, fmt.Errorf("unable to load secret immediately after creation %v: %v", p, err)
	}
	return s, nil
}

// createSecret will create the Secret, overwriting an existing secret if replace is true
func createSecret(ctx context.Context, s *fi.Secret, p vfs.Path, acl vfs.ACL, replace bool) error {
	data, err := json.Marshal(s)
	if err != nil {
		return fmt.Errorf("error serializing secret: %v", err)
	}

	rs := bytes.NewReader(data)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the wrapped cause and fix storage permissions (s3:PutObject / GCS objectOwner)
2. Use `kops replace secret -f <file>` after verifying write access with a test object
3. If the cause is 'serializing', check the fi.Secret input is well-formed (valid data, no unsupported fields)
4. Retry after transient storage errors; confirm with `kops get secret <id>`

Example fix

// before
err := store.ReplaceSecret("admin", sec) // unable to write secret: AccessDenied
// after: ensure write perms, then
if canWrite(store.VFSPath()) {
    _, err := store.ReplaceSecret("admin", sec)
}

When it happens

Trigger: Calling ReplaceSecret (or `kops replace secret`) when the VFS write to <secrets-basdir>/<id> fails: missing PutObject rights, object versioning/lock conflicts, or a serialization failure inside createSecret.

Common situations: Rotating the admin or dockerconfig secret with `kops replace secret` against a state store the operator can list but not write; S3 bucket policy changes after an incident lock-down.

Related errors

• error writing secret %q for mirror: %v
• unable to load secret immediately after creation %v: %v
• error loading secret %q: %w
• cannot parse SecretStore %q: %w
• secrets path is not cluster readable: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/a44611bb6a9209ce. Report an issue: GitHub.