argoproj/argo-workflows · error

failed to read secret

Error message

failed to read secret: %w

What it means

After optionally creating the 'argo-sso' cookie secret, newSso re-reads it with secretsIf.Get. If that Get fails (RBAC, not found, API error), construction aborts with 'failed to read secret' wrapping the kubernetes error.

Solutions

1. Inspect the wrapped error (NotFound vs Forbidden)
2. Grant the argo-server SA get permission on secrets in the namespace
3. Recreate/restart so the secret is re-created; stop whatever deletes argo-sso
4. Check namespace/pod disruption events around the startup time

Example fix

# before
# argo-server can create secrets but not read them
# after
kind: Role
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["create", "get"]

if _, err := sso.New(ctx, cfg, secretsIf, baseHRef, secure); err != nil {
    if strings.Contains(err.Error(), "failed to read secret") && apierrors.IsNotFound(err) {
        // secret was deleted mid-startup: safe to retry pod start
        return fmt.Errorf("argo-sso secret vanished, restart: %w", err)
    }
    return err
}

When it happens

Trigger: secretsIf.Get(ctx, secretName) returning an error during New() — secret deleted between create and get, get permission denied, or API server/transient failure.

Common situations: A mutating controller or cron job deletes the argo-sso secret; RBAC allows create but not get; kubernetes API briefly unavailable at pod startup.

Related errors

• failed to create secret
• failed to get service account secret
• failed to check if secret
• failed to list SSO RBAC service accounts
• failed to parse private key. If you have already defined a…

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/6f9e419305940fdc. Report an issue: GitHub.

Appendix: source

Thrown at server/auth/sso/sso.go:201

	}
	// whoa - are you ignoring errors - yes - we don't care if it fails -
	// if it fails, then the get will fail, and the pod restart
	// it may fail due to race condition with another pod - which is fine,
	// when it restart it'll get the new key
	_, err = secretsIf.Create(ctx, &apiv1.Secret{
		ObjectMeta: metav1.ObjectMeta{Name: secretName},
		Data:       map[string][]byte{cookieEncryptionPrivateKeySecretKey: x509.MarshalPKCS1PrivateKey(generatedKey)},
	}, metav1.CreateOptions{})
	isSecretAlreadyExists := false
	if err != nil {
		isSecretAlreadyExists = apierr.IsAlreadyExists(err)
		if !isSecretAlreadyExists {
			return nil, fmt.Errorf("failed to create secret: %w", err)
		}
	}
	secret, err := secretsIf.Get(ctx, secretName, metav1.GetOptions{})
	if err != nil {
		return nil, fmt.Errorf("failed to read secret: %w", err)
	}
	privateKey, err := x509.ParsePKCS1PrivateKey(secret.Data[cookieEncryptionPrivateKeySecretKey])
	if err != nil {
		if isSecretAlreadyExists {
			return nil, fmt.Errorf("failed to parse private key. If you have already defined a Secret named %s, delete it and retry: %w", secretName, err)
		}
		return nil, fmt.Errorf("failed to parse private key: %w", err)
	}

	clientID := clientIDObj.Data[c.ClientID.Key]
	if clientID == nil {
		return nil, fmt.Errorf("key %s missing in secret %s", c.ClientID.Key, c.ClientID.Name)
	}
	clientSecret := clientSecretObj.Data[c.ClientSecret.Key]
	if clientSecret == nil {
		return nil, fmt.Errorf("key %s missing in secret %s", c.ClientSecret.Key, c.ClientSecret.Name)
	}
	config := &oauth2.Config{

View on GitHub (pinned to 35bff19146)