argoproj/argo-workflows · error

failed to parse private key: %w

Error message

failed to parse private key: %w

What it means

Same ParsePKCS1PrivateKey failure, but in the normal path where argo itself just created (or freshly read) the argo-sso secret, so a corrupt pre-existing secret is not suspected; instead the freshly stored data failed to parse, which points to an unexpected mutation or corruption of the secret data.

Source

Thrown at server/auth/sso/sso.go:208

		Data:       map[string][]byte{cookieEncryptionPrivateKeySecretKey: x509.MarshalPKCS1PrivateKey(generatedKey)},
	}, metav1.CreateOptions{})
	isSecretAlreadyExists := false
	if err != nil {
		isSecretAlreadyExists = apierr.IsAlreadyExists(err)
		if !isSecretAlreadyExists {
			return nil, fmt.Errorf("failed to create secret: %w", err)
		}
	}
	secret, err := secretsIf.Get(ctx, secretName, metav1.GetOptions{})
	if err != nil {
		return nil, fmt.Errorf("failed to read secret: %w", err)
	}
	privateKey, err := x509.ParsePKCS1PrivateKey(secret.Data[cookieEncryptionPrivateKeySecretKey])
	if err != nil {
		if isSecretAlreadyExists {
			return nil, fmt.Errorf("failed to parse private key. If you have already defined a Secret named %s, delete it and retry: %w", secretName, err)
		}
		return nil, fmt.Errorf("failed to parse private key: %w", err)
	}

	clientID := clientIDObj.Data[c.ClientID.Key]
	if clientID == nil {
		return nil, fmt.Errorf("key %s missing in secret %s", c.ClientID.Key, c.ClientID.Name)
	}
	clientSecret := clientSecretObj.Data[c.ClientSecret.Key]
	if clientSecret == nil {
		return nil, fmt.Errorf("key %s missing in secret %s", c.ClientSecret.Key, c.ClientSecret.Name)
	}
	config := &oauth2.Config{
		ClientID:     string(clientID),
		ClientSecret: string(clientSecret),
		RedirectURL:  c.RedirectURL,
		Endpoint:     provider.Endpoint(),
		Scopes:       append(c.Scopes, oidc.ScopeOpenID),
	}
	idTokenVerifier := provider.Verifier(&oidc.Config{ClientID: config.ClientID})

View on GitHub (pinned to 35bff19146)

Solutions

1. kubectl -n argo delete secret argo-sso and restart argo-server so a fresh key is generated
2. Compare secret data to what argo writes: base64(DER PKCS#1 RSA key) under 'cookie-secret-key'
3. Look for mutating webhooks or backup/restore tooling that alters secret payloads
4. Check if the secret was created by a different argo-server version and whether any logs show it being modified

Example fix

# before
kubectl get secret argo-sso -o jsonpath='{.data.cookie-secret-key}' | base64 -d # garbage
# after
kubectl delete secret argo-sso -n argo && kubectl -n argo rollout restart deploy/argo-server

secret, _ := secretsIf.Get(ctx, "argo-sso", metav1.GetOptions{})
if secret != nil {
    if _, err := x509.ParsePKCS1PrivateKey(secret.Data["cookie-secret-key"]); err != nil {
        _ = secretsIf.Delete(ctx, "argo-sso", metav1.DeleteOptions{})
    }
}

if _, err := sso.New(ctx, cfg, secretsIf, baseHRef, secure); err != nil {
    if strings.Contains(err.Error(), "failed to parse private key") {
        _ = secretsIf.Delete(ctx, "argo-sso", metav1.DeleteOptions{})
        return fmt.Errorf("cookie secret corrupted; deleted for regeneration, restart argo-server")
    }
    return err
}

When it happens

Trigger: New() after create succeeded (or secret already existed but created in same code path without AlreadyExists), yet secret.Data['cookie-secret-key'] fails x509.ParsePKCS1PrivateKey.

Common situations: A controller/mutating webhook rewrote the secret's data (e.g. base64 re-encoding or field stripping); version mismatch between argo components where the secret format changed; storage corruption.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed to parse private key. If you have already defined a S
• failed to get service account secret: %w
• failed to create secret: %w
• failed to read secret: %w
• key %s missing in secret %s

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/7b3919b733e26dca. Report an issue: GitHub.