argoproj/argo-workflows · error

failed to create secret: %w

Error message

failed to create secret: %w

What it means

newSso tries to create the kubernetes Secret 'argo-sso' (secretName) holding the generated cookie-encryption private key. Errors are tolerated if the secret already exists (IsAlreadyExists), but any other failure (API server rejection, RBAC denial, quota, transient API error) aborts startup wrapped as 'failed to create secret'.

Source

Thrown at server/auth/sso/sso.go:196

		}
	}
	generatedKey, err := rsa.GenerateKey(rand.Reader, 2048)
	if err != nil {
		return nil, fmt.Errorf("failed to generate key: %w", err)
	}
	// whoa - are you ignoring errors - yes - we don't care if it fails -
	// if it fails, then the get will fail, and the pod restart
	// it may fail due to race condition with another pod - which is fine,
	// when it restart it'll get the new key
	_, err = secretsIf.Create(ctx, &apiv1.Secret{
		ObjectMeta: metav1.ObjectMeta{Name: secretName},
		Data:       map[string][]byte{cookieEncryptionPrivateKeySecretKey: x509.MarshalPKCS1PrivateKey(generatedKey)},
	}, metav1.CreateOptions{})
	isSecretAlreadyExists := false
	if err != nil {
		isSecretAlreadyExists = apierr.IsAlreadyExists(err)
		if !isSecretAlreadyExists {
			return nil, fmt.Errorf("failed to create secret: %w", err)
		}
	}
	secret, err := secretsIf.Get(ctx, secretName, metav1.GetOptions{})
	if err != nil {
		return nil, fmt.Errorf("failed to read secret: %w", err)
	}
	privateKey, err := x509.ParsePKCS1PrivateKey(secret.Data[cookieEncryptionPrivateKeySecretKey])
	if err != nil {
		if isSecretAlreadyExists {
			return nil, fmt.Errorf("failed to parse private key. If you have already defined a Secret named %s, delete it and retry: %w", secretName, err)
		}
		return nil, fmt.Errorf("failed to parse private key: %w", err)
	}

	clientID := clientIDObj.Data[c.ClientID.Key]
	if clientID == nil {
		return nil, fmt.Errorf("key %s missing in secret %s", c.ClientID.Key, c.ClientID.Name)
	}

View on GitHub (pinned to 35bff19146)

Solutions

1. Read the wrapped kubernetes error for the concrete API failure (403/422/etc.)
2. Grant the argo-server service account RBAC: create/get secrets in the namespace (see the sso-clusterroles)
3. Check for admission policies (OPA/Kyverno/PSA) blocking secret creation
4. Retry after transient API failures; if persistent, verify the namespace and namespace-scoped RBAC

Example fix

# before
# argo-server SA has no secrets create permission
# after
kind: Role
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["create", "get", "update"]

var secrets corev1.SecretList
if err := kubectl.AuthCanI(ctx, client, "create", "secrets"); err != nil {
    return fmt.Errorf("argo-server SA cannot create secrets: %w", err)
}

if _, err := sso.New(ctx, cfg, secretsIf, baseHRef, secure); err != nil {
    if strings.Contains(err.Error(), "failed to create secret") {
        return fmt.Errorf("check argo-server RBAC create on secrets and admission policies: %w", err)
    }
    return err
}

When it happens

Trigger: secretsIf.Create returning an error that is not AlreadyExists during New() — e.g. 403 Forbidden from RBAC, 422 validation, namespace not found, or a transient API server failure.

Common situations: Argo's service account lacks create/get on secrets in its namespace (restricted ClusterRole or Pod Security / secret RBAC policies); the argo-server namespace differs from the assumed one; API server outage.

Related errors

• failed to get service account secret: %w
• failed to read secret: %w
• failed to list SSO RBAC service accounts: %w
• failed to parse private key. If you have already defined a S
• failed to parse private key: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/364c815682627ef2. Report an issue: GitHub.